[linux-dvb] Re: Strange buffer overruns...

[Johannes Stezenbach <js@linuxtv.org>]

Josef Wolf wrote:
> 
> I experience strange buffer overruns.  I have:
> 
> - Called DMX_SET_BUFFER_SIZE to set 256*1024 byte buffer.
> 
> - Set up 16 PID filters (DMX_OUT_TS_TAP and DMX_PES_OTHER) to get 8 programs
>   from the dvr device.
> 
> - I have opened 8 files (one file for each program) to write the output
>   after I demux the contents.  Writes to the files occur nonblocking,
>   usually in 3kb..8kb chunks per write.

Writes to files are always non-blocking (unless you use O_SYNC or O_DIRECT)
because they go to the buffer cache, which is written to disk later.

> With this setup, top shows about 86% idle.  But about every five seconds
> I can hear the HD heads move, select() is blocked for about one second,
> and the read() from the dvr device reports EOVERFLOW.
> 
> My first assumtion was that flushing disks caused the overflows.  Then I
> tried to manually sync.  Strange enough, with this manually sync the disk
> was active for about two minutes without _any_ EOVERFLOWs while syncing.
> When sync was ready, the EOVERFLOWs came back again in 5 second intervalls.
> (I would have expected the opposite behavior).
> 
> My next experiment was to "while sleep 1; do sync; done".  With this command
> (run in a different terminal) the EOVERFLOWs disappeared completely.
...
>  unmaskirq    =  1 (on)

Maybe you should ask the file system gurus on lkml.

BTW, which chipset? For some old VIA chipsets it was well known that the PCI
bus is blocked during IDE transfers. You could try to increase the rate
at which the pdflush kernel thread is woken up (/proc/sys/vm/something);
if you use ext3 you could try to mount with commit=1 (see
linux/Documentation/filesystems/ext3.txt). Maybe there's a way to
limit the maximum length on a single IDE transfer.

Have you tried to raise the priority of your process? Maybe even use
a realtime policy?

Johannes