[linux-dvb] Re: Network: a new hope

[Emard <emard@softhome.net>]

On Sat, Nov 08, 2003 at 02:51:33PM +0100, Ralph Metzler wrote:
> Emard writes:
>  > While testing some more on network packet loss, 
>  > here I must review my statement about TS and PID bandwidth!
>  > 
>  > I think that so called 'section' code in dvb_demux.c is 
>  > not working 100% correctly and must be reviewed.
> 
> How exactly do you measure the packet loss?

First I increase sizes of IP rmem_* to cca 100 MB each, and netdev
backlog to 20000 packets. This is a must otherwise the kernel
wouldn't accept the incoming rate.

I take ethereal and scan complete dvb0_0 traffic for cca 10 seconds
on IP dedicated TS. Take it on peak hours (18-24h)

Then you need to pick-and-try some TCP stream that is long enough
to actually see something. Ccca 10-20 attempts are neeeded to pick
suitably long one, containing 100-1000 network packets. Some transmissions
are of course loss-free so don't get excited by a first lossless result.

After capturing dvb0_0 Just click on one TCP randomly (choose http or ftp)
and select Tools->TCP Stream Analysis->Time-Sequence Graph (Stevens).

Then look for discontinuity in VERTICAL direction (that is important for
data loss dicontinuty). The discontinuity in HORIZONTAL direction can
be ignored, as this is only a time lost discontinity (delay). You can 
sometimes see also multiple packets in HORIZONTAL direction, meaning the 
same packet is retransmitted again at a later time instance.

See my previous posts where I posted the actual snapshots and indicated
the exact points of loss. Just count the dots between two missing ones
and convert it to percentage. Each dot represents a packet of nearly 1500
bytes

> Can you determine exactly which packets are missing? With this I mean
> if you can see which TS packet seems to contain packets but is not
> demuxed as having an IP/MPE section.

If I took the TS of something predictable and outputting result, I could 
binary search in the TS and find it or at least locate neighbouring packets
that are before and after the missing one, so in the TS the missing one
should be located inbetween them most likely.

> I have a completetly rewritten section demux code. I test it in user
> space with the complete TS as input. It should be easy to see
> this way which packets are not handled correctly. 

To be honest, I am about to try some modifications of the demux code,
especially the one where it should finish the previous section and
continue the processing of new one where buf[p] is nonzero.

> But I do not see how this could have any influence on the 
> workings of the section demuxer. The 0x1fff packets are just skipped.

Let's suppose that dvb ip multiplex equipment sends < 0.5 bandwidth
data without trying hard to pack them, and as the bandwidth increases
it packs the data tightly, using more complex algorithms to fit each 
single byte in the available bandwidth.

> Or is the demuxer too slow and processes already overwritten data?
> But then this 0.5 TS bandwidth limit would depend on the CPU speed.

It seems that is efficiently written (on the expense of readability),
and it seems not to impose any significant load to CPU

Emard


-- 
Info:
To unsubscribe send a mail to ecartis@linuxtv.org with "unsubscribe linux-dvb" as subject.