[linux-dvb] video/audio mux and PTS

["Marcus O.C. Metzler" <mocm@convergence.de>]

Andreas Schultz writes:
 > Hi all,
 > 
 > One question for the dvb developers:
 > 
 > When trying to insert LPCM audio frames into a video stream send to the 
 > dvb card, several strange things occour (depending on how the stream is 
 > prepard).
 > 
 > The LPCM frames have been generated from an AC3 audio track of a DVD, 
 > the video frames come raw from the same DVD. On a DVD apparantly each 
 > AC3 frame has a PTS, the decoded LPCM data can be larger than 2022 
 > (2048-packet header). Only teh first frame gets the original PTS of the 
 > AC3 frame, any additional LPCM frames are therefore stuffed into the 
 > stream without a PTS. LPCM frames are created as PRIVATE_STREAM1, 
 > without the 0xBA packet header (PS packet?).
 > 
 > Video frames are used teh same way of the DVD, that is with a 0xBA 
 > header in front of the 0xE0 header. PTS are only present every second or 
 > so (every 25th frame).
 > 
 > This leads to stream where almost every audio frame has a PTS, but no 
 > SCR, and rare PTS in video frames, but each video frame has a SCR.
 > 

SCR is ignored by the firmware. It is only set at the beginning of
replay from the PTS information. After that synching is done with the
PTS information only. 

 > First question: Is the above describe stream OK and sensible so far?

Ok.


 > 
 > So, know the mysteries start, if i multiplex the stream so that audio is 
 > early (audio pts > video pts) or so that audio and video are synchron 
 > (audio pts == video pts), the video gets jumpy and audio is distorted 
 > sometimes. The later audio frames are (audio pts < video pts), the 
 > better the output gets (less jumpy), with a audio latency of about 1/2 
 > second (pts + 45000), the output is OK.
 > This makes me belive that audio should always be late by a certain value.
 > 
 > Question: How late should audio be? What are the max and min latencies 
 > allowed. Is there any rule or formula for this?
 > 

This always depends on the audio and video buffer sizes. For PCM you
need more space in the audio buffer per playing time than for MPEG
compressed sound, so that the buffer size for audio is in that sense
too small. This means audio data has to be send in a little later, so
that the audio data does not overflow the buffer while you are waiting
for the video frame.


 > ISO/IEC 13818-1, Section: 2.5.2.3 says the input buffer delay is max 1 
 > second, does this means that the maximum allowed audio frame latency is 
 > also 1 second? (This is supported by Section D.0.1 Timing Model cite: 
 > "therefore coded audio and video that represent sound and pictures that 
 > are to be presented simultaneously may be separated in time within the 
 > coded bit stream by as much as one second, which is the maximum decoder 
 > buffer delay that is allowed in the STD model.").
 > 

These specs are of course to set a limit on the needed buffer
sizes. In order to comply we would have to increase the audio buffer I
guess.


Marcus

-- 
---------------------------------------------------------------------
Dr. Marcus Metzler                             
mocm@netcologne.de                     http://www.metzlerbros.de
mocm@convergence.de                    http://www.convergence.de

Convergence Integrated Media GmbH          
Rosenthaler Str. 51                   
D-10178 Berlin                             
---------------------------------------------------------------------



-- 
Info:
To unsubscribe send a mail to listar@linuxtv.org with "unsubscribe linux-dvb" as subject.