IVS-C01-multimedia_applications_over_internet_v1.2.pdf

Internet Video Streaming Multimedia Applications over Internet Enrico Masala Version 1.2  Internet Philosophy parliamo principalmente di reti a pacchetto Internet offers a best-effort service Routes packets (on the basis of...

Internet Video Streaming Multimedia Applications over Internet Enrico Masala Version 1.2  Internet Philosophy parliamo principalmente di reti a pacchetto Internet offers a best-effort service Routes packets (on the basis of the destination Instrada i pacchetti (in base all' indirizzo di address only) as fast as it can destinazione) il più velocemente possibile Without considering the packet content, source address, applications senza considerate il contenuto del pacchetto, l'indirizzo e le applicazioni No processing inside the network Nessuna elaborazione all'interno della rete All Tutta application logic resides in devices at the edges la logica applicativa risiede nei dispositivi ai margini Opposite of traditional telephone network where devices at the edges are dumb IVS – © 2023 Enrico Masala 2  Potential Problems Main problems Packet losses i pacchetti possono andare persi Delay (and variability over time of delay) ritardi la variabilità nel tempo di questi ritardi Others that happens (less frequently) Reordering of packets Packet duplication pacchetti duplicati Bit errors IVS – © 2023 Enrico Masala 3  Packet losses Le code del router potrebbero essere piene questo può capitare quando si accumulano Router queues may be full pacchetti da smaltire in una destinazione è difficile da prevedere perché dipende da quello che fate voi ma anche da quello che Routers discard packets that cannot handle fanno gli altri This effectively reduces nominal bandwidth Since traffic arrives in queues from different sources it is difficult to predict the presence of congestion No notifications are sent about packet discard Packets are lost in bursts persi in blocchi di pacchetti consecutivi (i pacchetti che dovrebbero finire in quella coda vengono gettati via) sequence of consecutive packets; when a queue is full, it takes time before starting to accept new packets IVS – © 2023 Enrico Masala 4  Delay and Reordering Some time is needed before a packet reaches its destination Mainly due to waiting in router queues Highly unpredictable and variable over time (depends on the number of packets in router queues) Might cause packet reordering if packets follow different routes Cannot be lower than a given value due to tempo di propagazione propagation time (can be significant on long- distance communications) IVS – © 2023 Enrico Masala 5  Duplications and Bit Errors Less frequent Packets can be erroneously duplicated Due to retransmissions for missing acknowledgements at lower layers Due to malfunctioning network devices Packets can be corrupted by noise CRC or similar checks might miss corruption IP has no check for corruption of packet payload IVS – © 2023 Enrico Masala 6  Common Protocols Internet is based on the Internet Protocol (IP, RFC 791) Higher level protocols include sono i protocolli dominanti TCP (RFC 793) UDP (RFC 768) These are the protocol to use Because everybody uses them and ONLY them IVS – © 2023 Enrico Masala 7  TCP Creates point-to-point bidirectional //da ricordare bidirezionale connections for data transport Ensure data delivery in order without missing information Garantire la consegna dei dati in ordine senza informazione mancante Using acknowledgements and automatic retransmissions Utilizzo di riconoscimenti e automatici ritrasmissioni Automatically adapts the transmission rate to network and receiver conditions sono tutti meccanismi per gestire questa congestione Congestion control, flow control il ricevitore decide di non inviarli quando i dati si accumulano, quindi diciamo al trasmettitore di andare più piano IVS – © 2023 Enrico Masala 8  UDP servizio di trasmissione di pacchetti (è chi usa UDP) che ha deciso come fare i pacchetti ed ogni pacchetto è indipendente dall'altro Efficient protocol for datagram transmission Every transmitted packet is independent from the others Few functionalities provided Support to address specific applications (“demultiplexing”) by means of “ports” Checksum for the payload checksum per il carico utile All other aspects left to upper layers IVS – © 2023 Enrico Masala 9  TCP or UDP for Multimedia? Data sent over TCP is subject to delays which cannot be directly controlled by the application se c'è tanto traffico in rete e poca banda a disposizione, se c'è bisogno di poco ritardo TCP non è adatto Not suitable for typical low-delay multimedia communications, if congestion is present UDP leaves many implementation aspects to the application More suitable for low-delay multimedia communications in case of difficult network conditions (congestion, etc.) IVS – © 2023 Enrico Masala 10  IP Channel Characterization Depending on the protocol, Internet channel can be: Packet lossy channel (UDP) Packet loss rate (PLR): amount of packets lost (due to router discard of CRC failures) Variable end-to-end delay (minimum, average) la variabilità del ritardo sullo stesso canale di comunicazione Jitter: variation of the end-to-end delay due to queuing in the routers. The probability density function (p.d.f.) of the end-to- end delay is typically very useful to know Reliable channel with unpredictable bandwidth (TCP) canale assolutamente affidabile Bandwidth reduction translates into variable end-to-end delay NB: parameters may change over time! IVS – © 2023 Enrico Masala 11  How to Improve? Build on the existing platform Design new protocols with features useful for time-sensitive communications over packet lossy networks (e.g., RTP) Design new schemes aimed at improving performance for reliable but bandwidth-variable network (e.g., bitrate adaptation, multiple HTTP requests, …) Change the paradigm cambiare il paradigma di comunicazione Provide some form of quality of service (controlled packet loss ratio and delay) by changing the network architecture, i.e., routers (Diffserv (RFC 2475) and similar approaches) Can be applied only in fully controlled domains (ISP, large organizations) IVS – © 2023 Enrico Masala 12  Applications Requirements It is of paramount importance to understand the most common multimedia application scenarios to understand their requirements to counterbalance the issues that can be encountered in communication over packet networks per controbilanciare i problemi che possono (losses, jitter, bandwidth variations) essere riscontrati nella comunicazione su reti a pacchetto spesso si intende a qualche cosa a basso ritardo All multimedia applications are “real-time”, i.e., create and use packets that must be handled within predetermined amount of times Note that “real-time” meaning here is slightly different from common language IVS – © 2023 Enrico Masala 13  Application Classification Interactive c'è una comunicazione bidirezionale VoIP, audio/videoconferencing Unidirectional quelle che spesso chiamiamo col termine streaming “streaming” (watching movies, TV/radio etc.) Audio only ("radio") Audio/video Video only (videosurveillance) IVS – © 2023 Enrico Masala 14  Classification Live The source of multimedia data cannot be stopped (e.g., Television TV / Radio) Even in the case of pre-recorded content If data is captured from a source, data is compressed and sent in a small amount of time On demand Data is sent on request to a client that asks for them Data is already prepared for transmission and stored on some servers (e.g., YouTube, Netflix, Spotify, etc.), potentially after long optimization processes IVS – © 2023 Enrico Masala 15  General Requirement: Bandwidth avere banda a sufficienza per la trasmissione Every multimedia codec has an operating range Example: video MPEG-2 SD: ~ 2-6 Mbit/s Available bandwidth of the channel must fit in the operating range Otherwise, change codec, Quality discard some media or communication is impossible Rate IVS – © 2023 Enrico Masala 16  Bandwidth Variation Examples: MPEG-1: VHS quality: ~1.5 Mbit/s MPEG-2: SDTV quality: 2.5 – 6 Mbit/s MPEG-4: lower bitrates, higher flexibility [ See next slide for DVB-T bitrates ] Effect of bandwidth variation: IVS – © 2023 Enrico Masala 17  DVB-T TV “Digitale Terrestre”, Format: MPEG-TS MBAFF = interlaced with decision at the MB level (not at frame level) N=GOP size, M=Number of B frames +1; N. ref. frames is meaningful only for AVC E.g. M=3 N=12 IBBPBBPBBPBBI… Other audio channels: other languages, other formats NB: Observe “RAI”: AVC ~ same bitrate as MPEG-2 but much better resolution! [Other info: https://www.digitalbitrate.com/ also for some Satellite,Cable,OTT ] IVS – © 2023 Enrico Masala 18 General Requirement: Losses Limited but not necessarily zero! Voice communications (important case) Bit errors with < 1/1000 loss probability are typically transparent Frame loss probability (PLR) cipercetttivo da' abbastanza trasparenza dal punto di vista < 1% : typically transparent (“toll quality”) 3 – 5%: significant impact (e.g., more than 0.5 MOS reduction) >5%: unacceptable IVS – © 2023 Enrico Masala 19  Packet Losses / Bit Errors Missing data, for whatever reason (bit flips, packet loss, delayed data), causes issues Typical effects due to corruption of the MC-DPCM encoding in video: temporal and spatial propagation Consecutive frames ha dei dati erronei e produce qualcosa che non Error happen funziona bene IVS – © 2023 Enrico Masala 20  Packet Losses / Bit Errors Example: losses and how to manage them //0 di luminosità IVS – © 2023 Enrico Masala 21  General Requirement: Delay Delay is the main issue in interactive communications Sources of delay: il ritardo è da valutare dal punto di vista dell'applicazione Network Receiver Encoder Decoder buffer Delay is to be measured from the application point of view: mouth-to-ear, camera-to-eye < 150 ms: requirement for “toll quality”, traditional landline telephone service, ITU-T G.114 > 400 ms: generally unacceptable IVS – © 2023 Enrico Masala 22  Delay in the Transmission Chain End-to-end delay: Delay at the encoder Network delay (uplink, backbone, downlink) Resynchronization delay (playout buffer) and decoding Transmission delay la grandezza del ritardo dipende da come configuriamo il codificatore ENCODER DECODER Encoding Router Router Decoding delay delay Packetization Playout delay delay Propagation delay / queueing delay IVS – © 2023 Enrico Masala 23  Delay at the Encoder Delay at the encoder: tempo richiesto per codificare il frame Time required to encode the frame è un ritardo sia di processamento che di tipo algoritmico It includes both processing and algorithmic delay E.g.: IBBP scheme, 25 fps: Time required to encode first B in live scenarios since the frame is available: ex.: se abbiamo due frame di tipo B, vuol dire che abbiamo due tipi di frame da aspettare minimum 40*2=80 ms (algorithmic) + time to encode P (processing) – only the last one can be improved using a faster processor Packetization delay: conversion into packet For efficiency, more than one application unit can be inserted into one packet. In case of video, usually not IVS – © 2023 Enrico Masala 24  Delay Network delay Time needed to access physical medium E.g. in Multiple access systems Time needed to propagate signal e.g. Satellite links (geostationary satellite: 36000 km, speed of light in vacuum: 3*108 m/s, ~240 ms) e.g., 10000km fiber optic = 50 ms (~2*108 m/s) Cannot be reduced, minimum with non-congested network Time spent waiting in the router queues Highly variable component IVS – © 2023 Enrico Masala 25  Jitter Delay variation (jitter): Inter-packet gaps vary after crossing the Il gap tra pacchetti varia dopo aver network (time varying conditions) attraversato la rete (condizioni variabili nel tempo)Il gap tra pacchetti varia dopo aver attraversato la rete Packet reordering might happen in some (condizioni variabili nel tempo) networks (e.g. different paths) RTP sequence number helps to detect gaps/reordering and solve them Network Server IVS – © 2023 Enrico Masala 26  Playout Buffer Playout (dejittering) buffer To smooth the effect of unequal delay of packets Tradeoff between introduced delay and performance It also allows to reorder packets if needed This delay might be much greater than the others May be very large if content is pre-encoded (can be sent faster than real-time) Should be limited for interactive communications Network Playout buffer Client VoD server Introduces a delay IVS – © 2023 Enrico Masala 27  Interactive Applications For interactive applications, in order to achieve the so-called “toll quality”, the end-to-end delay must be lower than 150 ms (ITU-T G.114) In order to achieve this goal Codec: use a low-delay codec, which takes smaller chunks of data each time Packetization delay: lower the number of frames into a single packet, i.e., down to 1 Possible increase of overhead (e.g., 802.11 CSMA/CA ("WiFi") contention times) Minimize the size of the playout buffer IVS – © 2023 Enrico Masala 28  Decoding Delay Decoding delay Needed to reconstruct the multimedia signals once the compressed data of a multimedia unit Necessario per ricostruire i segnali multimediali are available una volta che i dati compressi di un'unità multimediale sono disponibili For each frame, it must definitely be lower than the amount of time represented by the frame, otherwise no real-time playback is possible Per ogni fotogramma, deve essere sicuramente inferiore a la quantità di tempo rappresentata dal fotogramma, altrimenti non è possibile la riproduzione in tempo reale IVS – © 2023 Enrico Masala 29 On Demand/Live Requirements On demand requisito ulteriore -> accesso casuale Random access, support by file formats By definition: single user Live un po' come quello delle applicazioni interattive, anche qui vorremmo l'accesso casuale ma abbiamo il tempo come tolleranza Possibility to connect after start (random access) Example: video, periodic insertion of I frames (e.g., interval from 0.5 to 3-5 seconds) (TV "zapping") Moderate end-to-end delay is acceptable (few s) Broadcast/multicast communication acceptable but prevent individual receiver feedback Typically not supported by generic Internet service IVS – © 2023 Enrico Masala 30  Legacy Services over IP Why telecom providers aim at moving all their legacy services (e.g., voice) to IP? Only one network to manage (IP infrastructure), needed in any case (Internet service) standardized open technology (IP): no vendor lock-in in a fully controlled domain: if needed, QoS can be guaranteed for legacy services e.g. via architectural solutions (DiffServ and similar) It is easy to introduce new features Example: security features (cryptography, authentication, …), wideband voice, etc. IVS – © 2023 Enrico Masala 31