Questions and Answers
What is the typical threshold for humans to perceive audio/video synchronization errors?
What is the McGurk effect?
Hearing Lips and Seeing Voices
What process involves embedding content and time references into a single data flow?
Transport Streams are mainly aimed at error-free applications.
Signup and view all the answers
In MPEG Systems, two clock values present in headers are PTS (Presentation Time Stamp) and ____. (Decoding Time Stamp)
Signup and view all the answers
Match the media container formats to their descriptions:
Signup and view all the answers
Study Notes
Synchronization in Internet Video Streaming
- Synchronization is crucial in media streaming to avoid a bad quality of experience, which can occur when audio and video are not in sync.
- Humans typically perceive audio-video synchronization errors if the difference in time ("skew") is higher than about 80 ms.
Audio and Video Synchronization
- To achieve synchronization, different media streams (audio and video) need to be synced using time references for specific points in media, such as video frames and audio frames.
- All content and time references need to be embedded into a single data flow, which requires data multiplexing.
Audio and Video Multiplexing
- MPEG standards provide a mechanism to achieve multiplexing with proper synchronization.
- Multiplexing is typically achieved through Time Division Multiplexing (TDM), where audio, video, and data are inserted as chunks in a single stream.
- These chunks are named "packets" in the MPEG standard.
Stream Types
- There are two types of streams: Elementary Stream (ES) and Program Stream (PS) / Transport Stream (TS).
- Elementary Stream (ES) is a sequence of encoded bits (video) or NALUs (since AVC) and sequence of encoded frames (audio).
- Program Stream (PS) is aimed at error-free applications, such as files on reliable storage, while Transport Stream (TS) is adapted from PS and aimed at error-prone applications, such as digital TV.
Packetized Elementary Stream
- Packetized Elementary Stream (PES) is a packetized version of ES, where each packet contains a header, timestamp, and data bits.
- PES packets can be further divided into groups called "packs".
PES Stream IDs
- PES streams can support up to 32 audio streams, 16 video streams, and other streams for information and control, such as private, proprietary, or program guide information.
Transport Streams
- Transport Streams (TS) have fixed-length (188 bytes) packets with headers derived from PES streams.
- These packets contain a transport header and stuffing bits that will be discarded.
- TS headers contain a master clock, typically 27 MHz, which is used for synchronization.
Synchronization
- TS headers contain a master clock (PCR) and two clock values: PTS (Presentation Time Stamp) and DTS (Decoding Time Stamp).
- PTS is the time at which decompressed data should be played back, while DTS is the time at which compressed data should be decoded.
Container Formats
- Container formats specify a binary syntax suitable to contain media data encoded with various codecs.
- Examples of container formats include MP4, AVI, MKV, MOV, VOB, 3GP, and ASF.
- These formats typically include additional metadata, such as seek support and indexing.
Mixed-Media Formats
- Mixed-media formats can contain audio, video, and metadata, such as MP4, AVI, and MKV.
- MP4 is a popular container format derived from MPEG-4 Part 12 (ISO Base Media File Format) and is widely used for online video streaming.
Matroska Multimedia Container
- Matroska is an open standard, open format, royalty-free container format (.mkv).
- It is based on Extensible Binary Meta Language (EBML) and is suitable for streaming, with support for out-of-the-box playback on Windows 10 and WebM.
Studying That Suits You
Use AI to generate personalized quizzes and flashcards to suit your learning preferences.
Description
Learn about the importance of synchronization in media streaming to avoid poor user experience, and how audio-video synchronization errors occur when timing is off.