Investigate what intermediary format should be used
Closed, ResolvedPublic

Description

We should support at most 2 intermediary formats one of them has to be open and royalty free.

thiblahute updated the task description. (Show Details)
thiblahute raised the priority of this task from to Normal.
thiblahute added projects: Pitivi 0.96, Pitivi.
thiblahute added a subscriber: thiblahute.
thiblahute moved this task from Backlog to In progress on the Pitivi 0.96 board.Oct 26 2015, 5:55 PM
thiblahute assigned this task to Mathieu_Du.
Mathieu_Du added a comment.EditedOct 26 2015, 5:55 PM

input video :

[ptv] [meh@meh-host pitivi-git]$ gst-discoverer-1.0 yopnosound.mkv
Analyzing file:///home/meh/pitivi-git/yopnosound.mkv
Done discovering file:///home/meh/pitivi-git/yopnosound.mkv

Topology:

container: Matroska
  video: H.264 (High Profile)

Properties:

Duration: 0:04:14.421000000
Seekable: yes
Tags: 
    container format: Matroska
    extended comment: MAJOR_BRAND=mp42
    encoder: Lavf56.25.101
    video codec: H264
    language code: und

[ptv] [meh@meh-host pitivi-git]$ ls -lh yopnosound.mkv
-rw-rw-r--. 1 meh meh 65M Oct 11 00:54 yopnosound.mkv

[ptv] [meh@meh-host pitivi-git]$ time gst-validate-1.0 uridecodebin uri=file:///home/meh/pitivi-git/yopnosound.mkv ! videoconvert ! fakesink --set-scenario scrub_forward_seeking_full
[snip]
real 1m38.009s
user 8m17.668s
sys 0m5.549s
[ptv] [meh@meh-host pitivi-git]$

jpegenc

[ptv] [meh@meh-host pitivi-git]$ time gst-launch-1.0 uridecodebin uri=file:///home/meh/pitivi-git/yopnosound.mkv ! jpegenc quality=100 ! matroskamux ! filesink location=output.mkv
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
Redistribute latency...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Got EOS from element "pipeline0".
Execution ended after 0:01:04.367388024
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...

real 1m4.646s
user 1m18.908s
sys 0m1.392s
[ptv] [meh@meh-host pitivi-git]$ ls -lh output.mkv
-rw-rw-r--. 1 meh meh 1.8G Oct 14 15:56 output.mkv
[ptv] [meh@meh-host pitivi-git]$

htop shows only one thread blasting at 100 %

[ptv] [meh@meh-host pitivi-git]$ time gst-validate-1.0 uridecodebin uri=file:///home/meh/pitivi-git/output.mkv ! videoconvert ! fakesink --set-scenario scrub_forward_seeking_full
[snip]
real 0m12.753s
user 0m24.588s
sys 0m1.954s
[ptv] [meh@meh-host pitivi-git]$

More jpegenc

[ptv] [meh@meh-host pitivi-git]$ time gst-launch-1.0 uridecodebin uri=file:///home/meh/pitivi-git/yopnosound.mkv ! jpegenc quality=85 ! matroskamux ! filesink location=output.mkv
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
Redistribute latency...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Got EOS from element "pipeline0".
Execution ended after 0:00:19.390166772
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...

real 0m19.663s
user 0m38.564s
sys 0m0.571s
[ptv] [meh@meh-host pitivi-git]$ ls -lh output.mkv
-rw-rw-r--. 1 meh meh 441M Oct 26 18:56 output.mkv
[ptv] [meh@meh-host pitivi-git]$ time gst-validate-1.0 uridecodebin uri=file:///home/meh/pitivi-git/output.mkv ! videoconvert ! fakesink --set-scenario scrub_forward_seeking_full
[snip]

> Test PASSED (Return value: 0)

real 0m5.390s
user 0m10.152s
sys 0m0.605s
[ptv] [meh@meh-host pitivi-git]$

This obviously raises the question of whether we want to render from the source clips or the proxy ones. If our preferred workflow is to render from the source clips, then having lower-quality proxies is acceptable.

thiblahute added a comment.EditedOct 26 2015, 6:15 PM

This obviously raises the question of whether we want to render from the source clips or the proxy ones. If our preferred workflow is to render from the source clips, then having lower-quality proxies is acceptable.

That would make no sense to do that as it brings in the same problem as we currently have... meaning we have to support the shitload of broken formats that exist out there at editing time.... this bug is good example of brokeness we will encounter doing that.

OK so here are the goals I see for this design / workflow:

  1. Allow for smooth editing at preview time
  2. Allow sidestepping multiple race conditions / issues during playback
  3. Allow sidestepping a subset of these issues during rendering

Rendering from source clips doesn't solve 3), but it solves 1) and 2)

One very important thing of the software from my point of view is to guarantee that the rendered video will exactly correspond to what the user edited, rendering from source videos makes guaranteeing that much more complicated which is why I consider we should not do that even if I totally understand that we will loose quality when rendering from the intermediary format.

jeff added a subscriber: jeff.Oct 29 2015, 2:02 PM

One very important thing of the software from my point of view is to guarantee that the rendered video will exactly correspond to what the user edited, rendering from source videos makes guaranteeing that much more complicated which is why I consider we should not do that even if I totally understand that we will loose quality when rendering from the intermediary format.

More complicated perhaps, but anything that involves forcing rendering the final output "from the proxies" is unacceptable in my humble opinion. A huge huge appeal of Pitivi to me was to avoid that. And other various commercial video editors that are "not finalcut" manage to do it too.

If you render from the proxies instead of from the sources, that means my footage gets encoded (and thus degraded) four times: by the camera, by the proxy/intermediate codec, by the rendered file, and then by the publisher (streaming video hosting service or TV broadcasting network). There's only so much codec damage an image can withstand...

In T3405#53291, @jeff wrote:

One very important thing of the software from my point of view is to guarantee that the rendered video will exactly correspond to what the user edited, rendering from source videos makes guaranteeing that much more complicated which is why I consider we should not do that even if I totally understand that we will loose quality when rendering from the intermediary format.

More complicated perhaps, but anything that involves forcing rendering the final output "from the proxies" is unacceptable in my humble opinion. A huge huge appeal of Pitivi to me was to avoid that. And other various commercial video editors that are "not finalcut" manage to do it too.

If you render from the proxies instead of from the sources, that means my footage gets encoded (and thus degraded) four times: by the camera, by the proxy/intermediate codec, by the rendered file, and then by the publisher (streaming video hosting service or TV broadcasting network). There's only so much codec damage an image can withstand...

Intermediarry formats should be (at least almost) lossless, we agree on that.

I just thought a bit about what audio format we should use and FLAC is probably the one we want to use. It is free an lossless :)

From my testing and previous discussions, this is what I think we should use:

If prores avalaible:

ProRes (HQ profile) and FLAC in Matroska

Otherwise:

Jpeg (quality = 100) and FLAC in Matroska

Anyone against that?

thiblahute added a comment.EditedNov 24 2015, 9:10 PM

OK, I came up with some simple benchmarking and the number makes me change my mind.

The benchmarking does 2 things:

  • Check how the output looks like after a big number of dec ! enc cycles (and the time it takes)
  • Transcoding metro.MOV (yes this one again xD) and run the scrub_forward_seeking validate scenario on it (and check time it takes)

And from my point of view, lossless, every frame is a keyframe h264 is the big winner here:

Encoder:: x264enc speed-preset=ultrafast qp-min=0 qp-max=0 key-int-max=1 
============
num 'decode ! encode':  0 -- duration: 0.09707784652709961s -- size: 1.1MiB
num 'decode ! encode':  1 -- duration: 0.17171978950500488s -- size: 1.1MiB
num 'decode ! encode':  10 -- duration: 0.9602432250976562s -- size: 1.1MiB
num 'decode ! encode':  20 -- duration: 1.999298095703125s -- size: 1.1MiB

Encoder:: avenc_prores 
============
num 'decode ! encode':  0 -- duration: 0.22787070274353027s -- size: 1.3MiB
num 'decode ! encode':  1 -- duration: 0.3574833869934082s -- size: 1.3MiB
num 'decode ! encode':  10 -- duration: 1.6417498588562012s -- size: 1.1MiB
num 'decode ! encode':  20 -- duration: 2.8870859146118164s -- size: 1.1MiB

Encoder:: jpegenc quality=100 
============
num 'decode ! encode':  0 -- duration: 0.11135673522949219s -- size: 1.8MiB
num 'decode ! encode':  1 -- duration: 0.18625640869140625s -- size: 2.1MiB
num 'decode ! encode':  10 -- duration: 0.991943359375s -- size: 4.7MiB
num 'decode ! encode':  20 -- duration: 2.1404917240142822s -- size: 4.8MiB
Testing: h264-lossless
========
  Encoding file:///home/thiblahute/devel/pitivi/benchmarks/intermediary_format/metro.MOV with: x264enc speed-preset=ultrafast qp-min=0 qp-max=0 key-int-max=1
     -- > duration: 1.597353458404541s -- size: 81.7MiB
  Scrub forward seeking on: /home/thiblahute/devel/pitivi/benchmarks/intermediary_format/outdir/metro.h264-lossless.mkv
     -- > duration: 2.4349007606506348s

Testing: prores
========
  Encoding file:///home/thiblahute/devel/pitivi/benchmarks/intermediary_format/metro.MOV with: avenc_prores
     -- > duration: 7.690262794494629s -- size: 150.9MiB
  Scrub forward seeking on: /home/thiblahute/devel/pitivi/benchmarks/intermediary_format/outdir/metro.prores.mkv
     -- > duration: 2.2498345375061035s

Testing: jpeg
========
  Encoding file:///home/thiblahute/devel/pitivi/benchmarks/intermediary_format/metro.MOV with: jpegenc quality=100
     -- > duration: 7.122682571411133s -- size: 197.0MiB
  Scrub forward seeking on: /home/thiblahute/devel/pitivi/benchmarks/intermediary_format/outdir/metro.jpeg.mkv
     -- > duration: 1.7318363189697266s

It has the best encoding speed, the best encoding ratio, and it sensibly slower to seek, but it is LOSSLESS so we do not care at all to use it instead of the actual sources to render \o/

thiblahute added a comment.EditedNov 24 2015, 10:16 PM

I check diffs of frame in h264 and min-qp max-qp was not the way to make actual lossless with h264, I had to set quantizer=0 pass=quant for it to be the case, and results are not that great actually :/

Encoder:: x264enc speed-preset=ultrafast qp-min=0 qp-max=0 key-int-max=1 quantizer=0 pass=quant 
============
num 'decode ! encode':  0 -- duration: 0.1298372745513916s -- size: 2.2MiB
num 'decode ! encode':  1 -- duration: 0.24788165092468262s -- size: 2.2MiB
num 'decode ! encode':  10 -- duration: 1.3177847862243652s -- size: 2.2MiB
num 'decode ! encode':  20 -- duration: 2.4290928840637207s -- size: 2.2MiB

Encoder:: avenc_prores 
============
num 'decode ! encode':  0 -- duration: 0.2805948257446289s -- size: 1.3MiB
num 'decode ! encode':  1 -- duration: 0.4480273723602295s -- size: 1.3MiB
num 'decode ! encode':  10 -- duration: 1.7831366062164307s -- size: 1.1MiB
num 'decode ! encode':  20 -- duration: 3.3797767162323s -- size: 1.1MiB

Encoder:: jpegenc quality=100 
============
num 'decode ! encode':  0 -- duration: 0.18667268753051758s -- size: 1.8MiB
num 'decode ! encode':  1 -- duration: 0.2110579013824463s -- size: 2.1MiB
num 'decode ! encode':  10 -- duration: 1.0583407878875732s -- size: 4.7MiB
num 'decode ! encode':  20 -- duration: 2.308377504348755s -- size: 4.8MiB
Testing: h264-lossless
========
  Encoding file:///home/thiblahute/devel/pitivi/benchmarks/intermediary_format/metro.MOV with: x264enc speed-preset=ultrafast qp-min=0 qp-max=0 key-int-max=1 quantizer=0 pass=quant
     -- > duration: 1.8500895500183105s -- size: 165.3MiB
  Scrub forward seeking on: /home/thiblahute/devel/pitivi/benchmarks/intermediary_format/outdir/metro.h264-lossless.mkv
     -- > duration: 3.808138370513916s

Testing: prores
========
  Encoding file:///home/thiblahute/devel/pitivi/benchmarks/intermediary_format/metro.MOV with: avenc_prores
     -- > duration: 7.537431240081787s -- size: 150.9MiB
  Scrub forward seeking on: /home/thiblahute/devel/pitivi/benchmarks/intermediary_format/outdir/metro.prores.mkv
     -- > duration: 2.1407902240753174s

Testing: jpeg
========
  Encoding file:///home/thiblahute/devel/pitivi/benchmarks/intermediary_format/metro.MOV with: jpegenc quality=100
     -- > duration: 6.723835706710815s -- size: 197.0MiB
  Scrub forward seeking on: /home/thiblahute/devel/pitivi/benchmarks/intermediary_format/outdir/metro.jpeg.mkv
     -- > duration: 1.7372910976409912s

And avenc_prores_ks:

Encoder:: avenc_prores_ks 
============
num 'decode ! encode':  0 -- duration: 0.2923121452331543s -- size: 1.4MiB
num 'decode ! encode':  1 -- duration: 0.46938562393188477s -- size: 1.4MiB
num 'decode ! encode':  10 -- duration: 1.9209380149841309s -- size: 1.3MiB
num 'decode ! encode':  20 -- duration: 3.3093013763427734s -- size: 1.3MiB
Testing: prores_ks
========
  Encoding file:///home/thiblahute/devel/pitivi/benchmarks/intermediary_format/metro.MOV with: avenc_prores_ks
     -- > duration: 25.516252040863037s -- size: 189.4MiB
  Scrub forward seeking on: /home/thiblahute/devel/pitivi/benchmarks/intermediary_format/outdir/metro.prores_ks.mkv
     -- > duration: 2.606330156326294s
thiblahute closed this task as Closed.Jan 18 2016, 2:54 PM
thiblahute moved this task from Needs review to Done on the Pitivi 0.96 board.

So the winner is the following combination ?

  • lossless, every frame is a keyframe H264
  • FLAC
  • MKV
thiblahute edited projects, added Pitivi (0.96); removed Pitivi.Apr 17 2016, 1:02 AM
thiblahute moved this task from Backlog to Done on the Pitivi (0.96) board.Apr 17 2016, 1:05 AM

GitLab Migration Automatic Message

This bug has been migrated to GNOME's GitLab instance and has been closed from further activity.

You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.gnome.org/GNOME/pitivi/issues/1631.