[linux-dvb] Thoughts on minimum dvr buffer size calculation?

[Jan Panteltje <panteltje@yahoo.com>]

OK guys, here is some thought for the minimum size of
the dvr buffer in   
dmxdev.h in the DVB driver.
It currently reads:
#define DVR_BUFFER_SIZE (10*188*1024)

The time between when the kernel writes dirty buffer
to disk is set by bdflush./
bdflush is always started as one of the earliest
process (before any file system).
(Type man bdflush).
ps -x | grep bdflush
 5 ?        SW     0:02 [bdflush]
Low PID hey?

The program that controls bdflush from user space is
/sbin/update (type update --help);
When I type 
update -d 
then I see:
1:  500 Max number of dirty blocks to write each time
bdflush activated
5:  3000 Time for data buffers to age before flushing


Now let us look at the data rate from the driver:
This is limited when for example using a SkyStar1 by
the on board hardware and firmware.
But in the case of a budget card, only by the tuner
specs.

I Have both a SkyStar1 and a wintv nova.
I never had problems recoding a full transponder (set
pid filter to 8192) with the SkyStar1   
probably because of its limitations (it reconstructs a
partially .ts).
But SkyStar1 failed on HDTV forum on hotbird with data
errors.
The guys from convergence wrote somewhere in the
mailing list 'no way' can a SkyStar1 do
HDTV, because of limit in hardware / firmware...
So I bough the wintv nova, it does full TS, no on
board processor.
It is specified at 42.5 Mbps.
http://www.hgspot.hr/katalog.aspx?tip=11

Here is the math:
When we suck at 42.5 MBps from /dev/dvb/adapter0/dvr0
for example by doing simply a
cat /dev/dvb/adapter0/dvr0 > filename.ts
then the driver will have to pass 42 500 000 bits /
second via the dvr device.
That is 5 312 500 Bytes / second (5.3If the buffers
are only guaranteed to be flushed once a second, then
it should be able to hold
5.3 MB.

The current buffer size = 10 x 188 x 1024 = 1 925 120
Bytes.
How often the buffers are flushed, and how much data
is written at the time is set by bdflush.
Normally when a user installs Linux, this will be set
by the bdflush default settings.
update -d
shows these.

I do not know exactly the mechanism used by bdflush,
but I do know I get this:
panteltje:~# cat /dev/dvb/adapter0/dvr0 >
/reiser/video/q1.ts
cat: /dev/dvb/adapter0/dvr0: Value too large for
defined data type
almost immediatly, nothing to do with file sizes:
-rw-r--r--    1 root     root     48902144 Feb  3
12:20 q1.ts

Looking up 'Value too large for defined data type'
shows this to be -EOVERFLOW
(I will leave in the middle here is '-EOVERFLOW' is
used in the right context.
Now you may think that came from 'cat', but it comes
from the driver, ad also 
kills any other stuff using the driver.

I traced it down by putting some printk() statement in
dmxdev.c
(just look for EOVERFLOW in dmxdev.c)
Feb  3 12:20:53 panteltje kernel: dmxdev: buffer
overflow len=65612 free=4096
Feb  3 12:20:53 panteltje kernel: WAS
dvb_dmxdev_ts_callback(): EOVERFLOW ret=-1

This set me searching in the wrong direction (tyoing),
until I realized there must be a malloc()
somewhere that allocated bufffer size.
That did lead to 
#define DVR_BUFFER_SIZE (10*188*1024)

I increased it to (100*188*1024)   and no more
problems.

I strongly suggest somebody who knows details about
how bdflush handles the buffers (and the kernel)
calculates the MINIMUM time between buffer read /
writes.

I could also fix the problem by reconfiguring bdflush:
update -1 1000
update -5 100 
but this effects the whole system!  MB/s).
 
When we go HDTV, or simply want to record a whole
transponder, then we will see even higher data speeds.
I have seen a PCI card advertised for HDTV with 56
Mbps.

There is also the issue of machine speed, I dunno
exactly how much effect that has if any.
I have done the test on Astra2 transponder 10773.00 h
symbolrate 22000, with PID filter   
set to 8192 (passes full ts) on a wintv nova PCI and a
950 MHz Duron.
Tuning with
http://panteltje.com/panteltje/satellite/xdipo.tgz
Then using cat from dvr device to a file.
test are 100 % repeatable, I jus put the old driver
value back, and so was the problem.

For those that use a budget card or other high speed
card (HDTV) and have mysterious aborts,
consider what I just wrote.
I will leave it at this. as for me the problem is
fixed with the buffer size modification.




		
__________________________________ 
Do you Yahoo!? 
The all-new My Yahoo! - What will yours do?
http://my.yahoo.com