Multimedia Applications: Streaming, VoIP, and Characteristics

• Multimedia applications involve audio and video, presenting distinct challenges and requiring specific techniques.

Video and Audio Characteristics

• Video has a high bit rate, ranging from 100 kbps to over 3 Mbps, varying with video quality.
• Audio has a lower bit rate than video.
• Audio glitches are more noticeable than video glitches.
• Both video and audio can be compressed at varying quality levels.

Types of Multimedia Applications

• Streaming stored audio and video,Udacity.
• Conversational voice and video over IP (VoIP), Skype.
• Streaming live audio and video, GATech graduation ceremony.

Streaming Stored Video

• Begins playing within seconds of data receipt.
• Interactive, allowing pause, fast forward, skip, and rewind.
• Continuous playout without freezing.
• Stored on a CDN or implemented with a peer-to-peer model.

Streaming Live Audio and Video

• Similar to streaming stored content but with differences.
• Involves many simultaneous users across different locations.
• Sensitive to delay, but a ten-second delay is generally acceptable.

Conversational Voice and Video over IP (VoIP)

• VoIP is phone service over the Internet.
• Often involves multiple participants.
• Highly delay-sensitive, where delays over 400ms can be frustrating.
• Loss-tolerant, with techniques to conceal glitches and the ability to ask for repeats.

How VoIP Works

• VoIP is transmitted over the Internet, which operates on a "best effort" basis without guaranteed delivery or timing.

VoIP Topics

• Encoding.
• Signaling.
• QoS (Quality of Service) metrics.

Audio Encoding

• Analog audio, represented as a continuous wave, is encoded into digital format using discrete data.
• Audio is encoded by sampling thousands of times per second and rounding each sample's value, a process called quantization.

Encoding Technique

• PCM (Pulse Code Modulation) takes 8000 samples per second with 8-bit values for speech and 44,100 samples per second with 16-bit values for audio CDs.
• Encoding schemes include narrowband, broadband, and multimode, each with different characteristics and tradeoffs.
• Audio, like video, can be compressed, but there are tradeoffs.

Signaling Protocols

• Used to set up and tear down calls.
• Responsible for user location, session establishment, session negotiation, and call participation management.
• VoIP uses signaling protocols like SIP (Session Initiation Protocol).
• VoIP faces signaling challenges due to interfacing with the PSTN.

QoS metrics for VoIP

• End-to-end delay.
• Jitter.
• Packet loss.

End-to-End Delay

• Includes encoding, packetizing, network, playback buffer, and decoding delays.
• Below 150ms is unnoticeable, 150-400ms is noticeable but acceptable, and over 400ms is unacceptable.
• VoIP applications often discard packets delayed beyond a threshold.

Jitter

• Jitter interferes with reconstructing the analog voice stream, leading to gaps in audio.
• Audio gaps should be kept below 30ms to maintain intelligibility.
• The jitter buffer smooths out delay variations by buffering and playing packets at a steady rate.
• Tradeoffs exist between longer buffers that reduce packet discard but add end-to-end delay, and shorter buffers that reduce delay but increase packet loss.

Packet Loss

• Packet loss is inevitable.
• TCP is not suitable for VoIP due to retransmission delays and congestion control impacts.
• VoIP protocols use UDP.
• Packet loss occurs if packets never arrive or arrive after their scheduled playout. Tolerance for packet loss can range from 1 to 20 percent, depending on the voice codec and other factors.

Methods of Dealing with Packet Loss:

• FEC (Forward Error Correction) transmits redundant data to replace lost data.
• Interleaving mixes audio chunks to minimize consecutive losses.
• Error concealment estimates lost audio packets based on neighboring snippets. FEC consumes more bandwidth and increases playout delay. Interleaving increases latency and is less useful for VoIP. Error concealment is computationally efficient but may not always be accurate.

Live/On-Demand Streaming Introduction

• Streaming media content accounts for 60-70% of Internet traffic.
• Enabled by increased bandwidth, efficient video compression, and Digital Rights Management.
• Types of streamed content: live (sports events, concerts) and on-demand (Netflix, YouTube).
• The constraint for live streaming is that there is not a lot of room for pre-fetching content.

Video Streaming

• The video content is first created, compressed, secured with DRM, and hosted on a server.
• Content providers use their own data centers or third-party CDNs for scalability.
• End-users download, decode, and render the video.
• Goal: reduce size by removing similar information without significantly compromising quality.
• Lossy compression offers higher bandwidth savings but cannot recover original quality.

Image Compression Steps (JPEG)

• Transform RGB image into YCbCr components.
• Divide the image into 8x8 blocks and apply Discrete Cosine Transformation (DCT).
• Compress the matrix of coefficients using a Quantization table (lossy step).
• Perform lossless encoding to store the coefficients.

Video Compression and Temporal Redundancy

• Encode the first frame (I-frame) as a JPEG and then encode the difference between frames (P-frame).
• Insert an I-frame periodically (e.g., every 15 frames) to avoid using the difference all the way, and is known as a Group of Pictures (GoP).
• Encode a frame as a function of the past and future I- or P-frames (B-frame).

VBR vs CBR

• VBR (variable bitrate) offers better image quality than CBR (constant bitrate) but is more computationally expensive.
• Video compression requires heavy computation, with specialized hardware encoders for real-time live streaming.

Transport Protocols

• Videos need to be decoded the client side, and can fail partially if data is lost
• TCP provides reliability and congestion control, making it suitable for video delivery
• The intial vision was to use specialized video servers that know the state of the client and can pause.

HTTP protocol.

• Use existing HTTP protocol, stateless
• Clients are intelligent, download
• Can use CDN's
• Easier to bypass middleboxes

Progressive Download vs Streaming

• Send an HTTP GET request for video.
• waste of network
• Video buffer at the client to store this content in memory
• client tries to pace it
• byte-range requests
• streaming without stalls.
• Accounts for variations by using playout buffer:
• video buffer is 5 seconds.
• full, the client will wait for it to get depleted.

Streaming is in two states:

• Filling state
• Steady state

How to Handle Network and User Device Diversity

• Multiple bitrates encoding because:
• device over
• network environments over

Bitrate Adaptation in DASH

• Client dynamically adjusts the video bitrate based on the network conditions and device type, Dynamic Streaming over HTTP or DASH where dynamic streaming signifies the dynamic bitrate adaptation.

Goal of Bitrate Adaptation:

• A good quality of experience (QoE) is usually characterized by the following:
• Low or zero re-buffering:
• High video quality
• low video quality variations
• low startup latency

Bitrate Adaptation Algorithms

• Signals to select the bitrate depends on,
• network Throughput
• video Buffer

Rate-Based Adaptation Mechanisms

• involves estimating the future bandwidth.
• continuous throughput is mapped to discrete bitrate. Reasons for an added Factor:
• Conservative in our estimate
• VBR-encoded
• transport-layer overheads.

Issues with Bitrate Adaptation

1. bandwidth is 5 Mbps for the first 20 seconds
2. Available

UNDER-Estimation with Rate-Based Adaption

DASH clients have an ON-OFF pattern in the steady state. This happens when the video client has the buffer filled up. And it is waiting for to deplete:

• the TCP connection reset the congestion window.
• two competing TCP-flows they would have gotten their fair share.

Buffer-Based adaptation

• The bitrate of is a function of the buffer occupancy is for a higher bandwidth the bitrate
• Using buffer-based adaptation can overcome the errors in bandwidth estimation that we saw in rate-based adaptation.
• But it has its own issues
• Low or zero re-buffering
• Large buffer to implement Bitrate Adaptation Algorithms