[vdr] Compressing VDR recordings without losing quality.

Niko Mikkilä nm at phnet.fi
Sat Oct 30 19:03:54 CEST 2010


Hi and sorry for this long post. I thought to share some of my personal
experiences and insights on re-encoding DVB recordings. I might also
have a few useful tips that are not always found in these discussions at
Linux-specific forums and mailing lists.

Feel free to skip this wall of text if you aren't interested in
processing DVB recordings. :)

First a quick response to the previous post.

On 2010-10-29 at 18:50 -0700, VDR User wrote:
> On Fri, Oct 29, 2010 at 10:59 AM, Tony Houghton <h at realh.co.uk> wrote:
> > The point of H264 is that it can achieve better quality for a better bit
> > rate. Although some quality will be lost by transcoding, you could
> > probably halve the file size without making a noticeable difference.
> 
> That sounds good when you read the datasheets but real world results
> are a bit different.  Also, what you're referring to is encoding
> comparisons all from a raw source - not mpeg2 vs. the same mpeg2
> reencoded in h264.  I can't stress enough that there is no magic to be
> had here for the reasons in my previous post.

While I agree with this statement in general, a 50 % bitrate reduction
is easily possible at visually transparent quality even when the MPEG-2
source is noisy and blocky. Although with x264 this is a bit harder if
the video is interlaced: encoding as interlaced is not (yet) as
efficient as progressive encoding and bobbing/double-rate deinterlacing
will also require more bitrate. Bobbing with a good filter is in my
opinion preferable because most realtime deinterlacing solutions that I
have encountered are pretty bad compared to the best software methods. I
guess the expensive hardware chips aren't much better than Nvidia and
Ati GPU deinterlacing implementations.


Benefits of filtering video

If larger changes to the video content such as noise reduction and
same-rate deinterlacing are tolerated, it's quite possible to get below
1000 kbps on average for a collection of SD video streams while
retaining very good visual quality compared to the originals. This gives
the additional benefit of making streaming possible over a 1 or 2 Mbps
uplink.


Deinterlacing solutions

It should be noted that good deinterlacing and good noise reduction are
more difficult to achieve than one might think. For (bob-)deinterlacing,
MPlayer/MEncoder has -vf yadif=1,mcdeint=0:0:10 (add ,framestep=2 for
same-rate deinterlacing), which gives acceptable results. This means
that the image won't shimmer much and that most sloped edges won't
appear jagged. Yadif and mcdeint are far from perfect on these points,
but they are still better than most filters. Yadif alone may be good
enough for casual viewing, but it shimmers (bobs up and down) and smears
details _a_lot_ compared to mcdeint. Nvidia's best "temporal adaptive"
deinterlacer in VDPAU also has similar issues although its edge-directed
interpolation is slightly better.

The only significantly better freely available deinterlacing solution
that I know of is TempGaussMC (TGMC) AviSynth filter combined with
NNEDI2/EEDI2/EEDI3. Although AviSynth is Windows-only software (it's
GPLed, but tied to VFW), it works perfectly on x86(-64) Linux platforms
with avs2yuv and Wine. Only filters that use GPU acceleration (fft3dgpu)
can't be used. The y4m output from avs2yuv can be piped directly to x264
and other encoders.

Here's a recent comparison clip that includes output from PureVideo
(Nvidia GPU), yadif, TDeint and TGMC:
http://forum.doom9.org/showthread.php?p=1433847#post1433847
I did some comparisons of my own and the PureVideo result reflects that
of VDPAU's "temporal spatial" deinterlacing method. Note that TGMC has a
strong denoising and local stabilization effect that is useful in
compression. yadif+mcdeint would probably stand between TGMC and TDeint
in a quality comparison, but it's a few orders of magnitude faster than
TGMC+NNEDI2. However, unlike MEncoder filters, AviSynth filters can be
split evenly to multiple threads so AviSynth may be faster on a
multi-core CPU if only one encode is running at a time. If there's
enough RAM available (~ 128-512 MB for each SD AviSynth job or 64-128 MB
for MPlayer/MEncoder), it's easiest and most efficient to run as many
encoding jobs as there are CPU cores.


Denoising and deblocking

The best denoiser in MPlayer/MEncoder is hqdn3d, but it will smooth
details out along with the noise. Instead I've been using
motion-compensated MDegrain2 from AviSynth's MVTools2. It obliterates
most types of grain and noise but still gives a very natural and
non-plastic output. This makes further compression much easier. With
noisy film sources the bitrate can often be dropped to half from what it
would be without denoising.

I'm not going much into deblocking here because I haven't compared the
available methods myself. There are many filters available especially in
MPlayer/MEncoder. Deblocking should be coupled with the decoder so that
information on macroblock quantizers is available. Otherwise it will
smooth out actual detail in the video. Oversmoothing is an issue with
the good filters too.


Tradeoffs

As Lars Bläser pointed out earlier, all this processing and encoding
comes at a high price in CPU time and energy consumption. Whether it's
worth it depends on your purposes, so make your own calculations. If you
only want a bitrate reduction for storage reasons, CPU time alone may
cost more than the required disk space -- unless it gets cold outside
and you are heating with electricity...

Efficient encoding still requires lots of manual work because there are
no reliable automatic detection schemes for recognizing different types
of sources: interlaced, telecined, field-blended and phase-shifted.
Personally I'd like a scheme where the encoder frontend would adapt to
source type changes on the fly and encode the video with a variable
framerate, but this is not always desirable.


Scripts

I wrote a bash script for downloading and installing AviSynth plus some
of the essential filters on Linux. I could upload it somewhere if people
are interested? There's also a simple command-line based template system
that makes using AviSynth slightly easier, but it needs more work to be
useful.


Cheers,

Niko Mikkilä





More information about the vdr mailing list