[vdr] Re: VDR 1.3.12 buffer optimizations - SOLVED!

[Klaus Schmidinger <Klaus.Schmidinger@cadsoft.de>]

Well, looks like all the work I've done during the past
weeks has finally payed out :-) I have now modified VDR's
buffer handling in such a way that, on my machine (which
isn't all that fast) I can have 5 parallel recordings on
three DVB cards, plus a Transfer Mode or replay without
any problems!

Sorry that there is no full new version today, just a patch,
but this really needs to be tested thoroughly first. So I'm
offering a patch against VDR 1.3.12 for those who are willing
to try it at

  ftp://ftp.cadsoft.de/vdr/Developer/vdr-1.3.12-optbuf3.diff

Note that I had to make some changes to cRemux, cCondVar and
several other places, so there's a good chance some plugins
won't compile after this patch. But since my main goal is
improving VDR's recording capabilities, plugins are currently
none of my concern.

Here's the excerp from the HISTORY file that lists all changes
made for improving the buffer handling:

- Added cRingBufferLinear::Read() to read directly from a file handle into the
  ring buffer.
- Using timeouts in ring buffers to avoid 'usleep()'.
- Clearing the 'Transfer Mode' ring buffer after clearing the device to avoid
  an "almost full" ring buffer.
- Removed locking from cRingBufferLinear for better performance under high load.
- Using a cRingBufferLinear in cRemux to avoid unnecessary copying of data.
- Using a cRingBufferLinear in cTSBuffer and filling it in a separate thread
  to avoid buffer overflows. Plugins using cTSBuffer will need to remove the
  call to the now obsolete Read() function (see cDvbDevice::GetTSPacket() for
  the new usage of cTSBuffer).
- cRemux::Process() has been split into Put(), Get() and Del() to allow for a
  better decoupling of the remuxing and disk writing process. Plugins using
  cRemux will need to be modified accordingly.
- The actual disk writing in recordings is now done in a separate thread to
  improve the overall throughput.
- Changed cRemux so that it returns the maximum available amount of
  data with each call, not just 2048 byte.
- Added a visual display of all cRingBufferLinear buffers for debugging. To
  activate it, define DEBUGRINGBUFFERS in ringbuffer.h.
- Changed cCondVar to avoid a possible "near miss" condition (thanks to Sascha
  Volkenandt for pointing out this one). cCondVar now uses its own mutex and
  thus has a different calling interface. Plugins using cCondVar will need to
  be modified.
- The cRingBuffer now does EnableGet()/EnablePut() only if the buffer is more than
  one third full or empty, respectively. This dramatically improves recording
  performance and reduces system load (thanks to Marco Schlüßler for doing some
  testing regarding buffer performance and giving me some hints that finally lead
  to finding out that this was the basic problem causing buffer overflows).

Note that right now the Transfer Mode is a little "jerky" at the beginning,
and under some circumstances that I haven't found out yet, the Recording and
Result buffer run full immediately at the beginning of a recording. However,
this is most likely a problem that has nothing to do with the general data flow,
but rather with cRemux not being able to sync on the data. Any input regarding these
two problems is, of course, greatly appreciated :-)
Also, I'd like to hear whether this patch actually improves things for _your_
systems, too.

Klaus