Digital Video PDF

Digital Video Created @November 11, 2022 9:43 AM Course Multimediální technologie Procedure 5. Utilized Related - Incoming Related - Outgoing Status...

Digital Video Created @November 11, 2022 9:43 AM Course Multimediální technologie Procedure 5. Utilized Related - Incoming Related - Outgoing Status Asset Summary: Film/TV/Video sequence of images creating ilusion of motion film = motion picutre = art form and medium approx 130 years old phenomenon film reels - now digital video = electronic medium television = medium broadcasting video video or film does not always mean sound Film film not used like in photography - vertical (except IMAX) positives used in cinemas grain the random optical texture of processed photographic film due to the presence of small particles of a metallic silver or dye cloud Digital Video 1  difficult to store - fire, mold, gets lost archived by NFA - Czech Film Archive digitalization by scanning - both audio and video Video in 2020 video streaming: 57.64 % traffic share Youtube: 15.94 % traffic share Netflix: 11.42 % traffic share audio: 0.39 % Youtube 5+ billion videos 500 hours of videa uploaded every minute 2 billion registered users Digital Video 2  over billion hours watched daily US: streams also TV (ABC, Fox, CNN,…) Netflix 192.95 million subscribers 1.1 billion hours watched daily approx. 5600 shows Other HBO Max, HBO Now Amazon Prime Video Disney+, Hulu iVysílání Framerate = # of frames per second the more, the better smoother motion less stressed brain requires higher bitrates film 24, PAL TV 25/50, NTSC 29,97/59,94 frame = whole picture at the time field = only either odd or even lines Progressive, Interlaced, Pulldown Scan Progressive (P) a format of displaying, storing or transmitting moving images in which all the lines of each frame are drawn in sequence it appears smoother and more realistic Interlaced (I) - video fields Digital Video 3  traditional analog television only the odd lines and then the even lines of each frame are drawn alternately → only half the number of actual image frames are used to produce video Not used anymore in new standards Progressive segmented (PsF) - 1 frame is split into 2 fields Progressive image in interlaced transport chain 3:2 pull down = film is slowed to 29.97 fps and each 4 frames are transformed to 5 post-production process of transferring film to video film - 24 fps → transform to television standard - 30 fps Bottom line: Nowadays, only progressive HFR = high frame rate higher frame rates than typical prior practice 24 fps is motion picture standard 1992 - Momentum - IMAX HD - 48 fps 2012 - Hobbit - 48 fps 2016 - Billy Lynn's Long Halftime Walk - 120 fps, 4K and 3D DCI - recommends for 4K2D and 2K3D 48 and 60 fps, 2K2D adds 96 and 120 fps Digital Video 4  new Avatar and Animal Farm will be HFR UHDTV-2 specifies 120fps video file recording encoding: real-time method, menial method Shutter Speed time of exposure of single frame fraction of a second best to synchronize with frame rate defines the smoothness of motion expressed as an angle intertwined with frame rate spcifies how much of rotary disc shutter is open 360° = the shutter is fully open, shutter speed is the same with the time of 1 frame, i.e. 1/(frame rate) second Digital Video 5  Aspect Ratio the ratio of the width to its height it is expressed with 2 number movie industry silent film - 4:3 Talkies - 1.375:1 = academy ratio later also 4:3 because of TW super 35 - wider because the sound track was moved there are movies with multiple releases in different aspect ratios - on the waterfront: 1.66:1, 1.33:1, 1.85:1 TV and Digital video 4:3, 16:9 - HD and UHD - cameras 17:9 21:9 and more - some TVs, smartphones, monitors Photography still photography - 3:2 Bit-Depth the number of bits used to indicate the colour of a single pixel Digital Video 6  the number of bits used for each colour component of a single pixel before HDR only 8-bit, 10-bit in professional environment now 8-12-bit 8-bit - 256 colours 12-bit - 4096 colours even 8-bit source is better to encode as 10-bit + more effective compression and less colour banding - less compatible, higher complexity to encode and decode Limited range - TV users does not use all 8 bits of values 8b = 16-235, 10b = 64-940, 12b = 256-3760 Computers use whole range Rec. 709 standard specifying SDR HDTV for image encoding and signal charachteristics successor of Rec. 601 for digital recordings of interlaced analog video resolution 1920×1080p/i 23.976 up to 60 fps colour depth 8- and 10-bit colour space covering approx. 35.9 % of all colours no HDR RGB, 4:4:4, 4:2:2 a 4:2:0 Rec. 2020 = standard specifying SDR UHDTV with WCG Digital Video 7  resolutions 3840×2160 and 7680×4320 23.976 up to 120 fps colour depth 10- and 12-bit colour space covering approx. 76% of all colours no HDR RGB, 4:4:4, 4:2:2 and 4:2:0 supports HDMI 2.0, DP 1.4, UHD Blu-ray H.264, H.265, VP9, DNxHD Rec. 2100 = standard adding HDR to Rec. 2020 PQ and HLG adds resolution 1920×1080 adds new proprietary colour space ICTCP from Dolby SDR Video = standard dynamic range maximum brightness 100 cd/m² 8-bit - 6 stops, 10-bit - 10 stops SD, HD and UHD video and TV - resolution does not matter SD = standard definition still good for streamin - 480p HD = high definition HD vs SD (for streaming) - depends on on the bandwidth gamma curves and colour spaces Rec. 601, 709, 2020 HDR Video = high dynamic range enables higher brightness than 100 cd/m² Digital Video 8  ≥ 10-bit video resolution does not matter few competing PQ standards and HLG EOTF and OETF = electro-optical transfer function EOTF transfer function having the picture or video signal as input and converting it into the linear light output of the display Describes how to transfer digital data to light 2 EOTF systems PQ = perceptual quantizer - SMPTE ST 2084 a transfer function that allows HDR display by replacing the gamma curve used in SDR the basis of HDR video formats (Dolby Vision, HDR10…) HLG = hybrid log-gamma - ARIB STD-B67 a transfer function developed for HDR display it is backward compatible with the transfer function of SDR both in DVB/ETSI TS 101 154 - digital TV standard OETF Digital Video 9  transfer function having the scene light as input and converting into the picture or video signal as output for cameras PQ and HLG control HDR from recording, post-production to playback It is necessary because devices in video production flow are different, mainly TVs are not as good as color grading monitors PQ uses metadata created during post-production, HLG uses information about display and current light conditions PQ and Metadata Standards nonlinear transformation functions, absolute, more common for motion pictures up to 10000 nits, 28 stops HDR10 - static metadata Digital Video 10  high-dynamic-range video standard not backward compatible with SDR static metadata = information about the whole video HDR10+ - dynamic metadata, backwards compatible dynamic metadata - specific for each frame or each scene of the video used to adjust and optimize each frame of the HDR video to the consumer’s display capabalities both standards are open Dolby Vision - proprietary licenced standard dynamic metadata, backwards compatible with HDR10 static metadata supported by HDMI 2.0b, DP1.3, UHD Blu-ray dynamic only in HDMI 2.1, DP 1.4 HLG created by BBC and NHK open standard without royalties usually 1000 nits, but allows more backwards compatible with SDR devices suitable for TV broadcast HLG does not have metadata, it uses gamma curve supported in HDMI 2.0b, H.264, H.265, VP9 Adobe CC, Final Cut Pro X, Media Composer Compatibility streaming services: Amazon Video HDR10, HDR10+, Dolby Vision Digital Video 11  Youtube (upload) - HLG and HDR10 in H.264, VP9 Profile 2, ProRes and DNxHR Netflix, iTunes, Vudu, Google Play Movies HDR10 and Dolby Vision SOG 2016 were broadcasted in HDR in USA US DirecTV and UK Freview support HLG x264, x265, VP9, AV1 support all standards Windows, Linux, macOS support HDR10 most compatible, HDR10+ newest, but open, Dolby Vision supported, but proprietary marketing creates mess 3D Video = stereoscopic not really 3D, just 2D videos in past anaglyph glasses, nowadays either polarized or active shutter since the begining of movie era, popular in waves, IMAX 3D 2003 - Ghosts of the Abyss - Cameron first digital 3D movie 2009 - probably the best 3D movie - also Cameron 2 synchronized cameras with semi-translucent mirror, mounted perpendicularly nowadays many movies only converted from 2D now popular because of VR Motion sickness Digital Video 12  distribution on HD Blu-ray compressed in H.264 MVC - Multiview Video Coding 2D + delta of the second angle approx. 50 % higher bitrate distribution for download SbS - side by side - vertical TaB - top and bottom - horizontal both anamorphic HDCP protection against piracy DP, HDMI and DVI all devices in the video signal flow must support up to version 2.1 - hacked, backwards compatible version 2.2 - can be downgraded, not backwards compatible the more devices in the flow, the higher the probability of a bug Compression Principles Digital Video 13 Digital Video 14  Lossy Video Compression Reduction of spatial and temporal redundancy and irrelevance Intra data reduction takes place between the frames it exploits redundancy between successive frames = temporal redundancy it requires more processing time because the computer needs to look at several frames compression within single frame just like JPEG prediction transformation quantisation entropy coding inter frame codecs: H.264-A variant of MPEG-4 MPEG 4 MPEG 2 AVCHD-A variant of H.264 AVC Inter data reduction takes place within the frames it exploits redundancy within the frames Digital Video 15  = spatial redundancy it requires less time since the computer is dealing with single frame Across frames only temporal changes are recorded references to different frames forward and backward motion compensation - motion vectors only prediction error is saved the better the prediction, the less data necessary intra frame codecs: MJPEG-JPEGS Prores Cinema DNG DN*HD Other tools Slices - frame is sliced into multiple independent slices Tiles - frame is divided into independent horizontal rectangular regions Deblocking filter - removal of blocking artifacts coding tree - frame is divided into blocks SAO - Sample Adaptive Offset filter - improves single colour areas and edges many other in more complex formats Types of Frames a video frame is compressed using different algorithms with different advantages and disadvantages Digital Video 16  keyframe it defines the starting and ending points of smooth transitions inbetweens intermediate frames, between two keyframes I-frame intra frame - intra compression keyframe, anchor frame the least compressible don’t require other video frames to decode like a complete image, like a JPG or BMP image file P-frames predicted frame only changes in comparison to previous frame intra + inter compression anchor frame can use data from previous frames to decompress more compressible than I-frames B-frames bidirectional frame only changes in comparison to previous or subsequent frame decoder needs to first decode frames which will be displayed after the B-frame requires larger buffer, increases latency Digital Video 17  GOP - Group of Pictures frames between two I-frames defines by two numbers M - the distance between anchor frames N - size of GOP, distance between keyframes IBBBBPBBBBPBBBBI - M=5, N=15 Motion Vectors pieces which moved since last keyframe are recorded as a vector 100 % fit is improbable this difference is encoded = prediction error Video Coding Formats and Standards profiles - set of ecnoding/decoding capabilities to standardize capabilities of devices standardizes implemented algorithms, bit-depth, subsampling levels - set max resolution and bitrate enables release of cheaper HW Professional (Intermediate) Coding Formats Apple Prores White Paper proprietary lossy format, only I-frames,.mov up to 8K, ? fps, 4:4:4:4, 10-12-bit, 16-bit alpha channel Avid DNxHD/DNXHR a lossy high-definition video post-production codec HD = ST 2019-1:2008 - VC-3 Digital Video 18  only I-frames up to 1080i/p, 60fps, 4:4:4, 10-bit HR - for resolutions 2K+ up to 8k, 59.94 fps, 4:4:4, 12-bit Prores RAW White Paper debayering necessary 12+ bit-depth RAW a RAW HQ - 2 levels of lossy compression similar bitrate like ProRes 422 and ProRes 422 HQ metadata ISO, shutter speed, scene, take... Atomos recorders (not the one we have) Blackmagic RAW new and very modern codec Open but not open-source -.braw up to 12K, 12-bit possible to save proxy directly in camera -.sidecar metadata - ISO, shutterspeed, scene, take... partial debayering in camera - requires less computation during postproduction uncompressed, 3, 5, 8 and 12:1 compression, constant quality Q0 and Q5 can be processed by CUDA, OpenCL (GPU) and SSE, AVX (CPU) BMPCC 4K and 6K, URSA Mini Pro G1, G2 and 12K Premiere Pro, DaVinci Camera Raw Image File - contains unprocessed or minimally processed data from the image sensor of either a digital camera or a motion picture film scanner Digital Video 19  Delivery Coding Formats MPEG, VCEG groups developing proprietary coding standards MPEG The Moving Picture Experts Group established by ISO, IEC they set standards for media coding, including compression coding of audio, video, graphics … VCEG a working group of the ITU standards for compression coding of video, images, audio and other signals responsible for H.26x line of video coding standards or JPEG = ITU-T Video Coding Experts Group - approx. 1984 80's - ITU-T H.120 - for videotelephony 1990 - ITU-T H.261 =ISO/IEC Moving Picture Experts Group - 1988 1992 - ISO/IEC 11172-2 - MPEG-1 part 2 also MP3 = ISO/IEC 11172-3? - MPEG-1 part-3 1996 - together created ITU-T H.262/ISO/IEC 13818-2 - MPEG-2 part 2 ITU-T H.263 for cell phones and ISO/IEC 14496-2 - MPEG-4 part 2, Visual - used by DivX/XviD 2003 - ITU-T H.264/ISO/IEC 14496-10 - MPEG 4 part 10, Advanced Video Coding 2013 - ITU-T H.265/ISO/IEC 23008-2 - MPEG-H part 2, High Efficiency Video Coding 2020 - ITU-T H.266/ISO/IEC 23090-3 - MPEG-I Part 3, Versatile Video Coding Digital Video 20  homework!: https://jaroslavsvoboda.eu/vyuka/multitech/6.html#/33/1 VPX Family 1992 - The Duck Corporation Codecs TrueMotion - S, RT, 2 Mainly used in games - FFVII 1999 - renamed to On2 2002 - VP3 - opened and later released as Theora 2001 - 2005 - VP4, VP5 2005 - 2008 - VP6, VP7 - Macromedia Flash Video and VP7 also Skype 2010 - acquisition Google 2008 - VP8 - RFC6386 - opened 2012 - VP9 - open instead of VP10 Alliance for Open Media creates AV1 Digital Video 21  ISO/IEC 11172-2 - MPEG-1 PART 2 the only coded usesing D frames - low quality frames for preview during scrubbing VHS quality 352×240 @1.86 Mb/s up to 4095×4095, 100 Mb/s used by VCD, DVD-Video SW codec mpeg1video ITU-T H.262/ISO/IEC 13818-2 - MPEG-2 PART 2 16×16 makroblocks 8×8 blocks Multiview Profile (MVP) for 3D, interlaced 7 profiles and 4 levels up to 1920×11522, 60fps, 4:2:2, 8-bit, 80 Mb/s Used by DVB-T/S, DVD-Video, HDV, XDCAM, Blu-ray SW codecs: x262, MainConcept, Nero, mpeg2video... ITU-T H.264/ISO/IEC 14496-10 - MPEG 4 PART 10, ADVANCED VIDEO CODING 16×16 makroblocks 16×16−8×8 blocks Multiview Video Coding (MVC), interlaced 22 profiles and 20 levels Up to 8192×4320, 120fps, 4:4:4, 14-bit, 3200 Mb/s Used by DVB-T/T2/S2, Blu-ray, YouTube, Netflix, iVysílání, AVCHD, XAVC SW codecs: x264, OpenH264, MainConcept, DivX, Nero, Elemental... Digital Video 22  ITU-T H.265/ISO/IEC 23008-2 - MPEG-H PART 2, HIGH EFFICIENCY VIDEO CODING 64×64−8×8 CTUs Quadtree structure MVC, monochrome, HDR, WCG MANY profiles, 13 levels and 2 tiers: Main and High Up to 8192×4320, 120fps, 4:4:4, 16-bit Used by DVB-T2, UHD Blu-ray, Netflix SW codecs: x265, SVT-HEVC Kvazaar, openHEVC, Turing, DivX, Nero, MainConcept... Analýza kodeku High Efficiency Video Coding VP8 IETF RFC 6386 an open and royalty-free video compression format 16×16 makroblocks 4×4 blocks Adds Hadamard transform to DCT 1 profile and level up to 16384×16384, ∞fps, 4:2:0, 8-bit, alpha channel Used by YouTube and Wikipedia Digital Video 23  SW codecs: libvpx Alpha VP9 Bitstream specification 0.6 64×64 superblocs 32×32 DCT blocks HDR, WCG, transparency 4 profiles and 14 levels up to 65536×65536, ∞fps, 4:4:4, 8-12-bit Used by YouTube, Netflix, Wikipedia SW codecs: libvpx, SVT-VP9, EVE Analýza open format videokodeku AV1 Bitstream specification 1.0.0 128×128 − 64×64 superblocks 128×128 − 4×4 blocks HDR, WCG, transparency 3 profiles and 14 levels up to 16384×8704, 300 fps, 4:4:4, 8-12-bit, 800 Mb/s approx. 30 % better than VP9 Used by YouTube, Netflix, Vimeo and Facebook SW codecs: libaom, rav1e, SVT-AV1, dav1d Other AVS, AVS2 and AVS3 - Chinese formats, SW codecs: xavs a xavs2 GoPro CineForm - similar to ProRes and DNxHD Ut Video Codec Suite - lossless Digital Video 24  Theora - Open royalty-free codec Xvid - open-source version of first DivX - MPEG-4 part 2 Microsoft: SMPTE 421M-2006 VC-1 - Blu-ray , WMV2 - Windows Media Video 8 BBC: Dirac or VC-2 - open-source broadcast intermediate format Y4M - container for uncompressed video Bink, Cinepak, FFV1, RealVideo... Encoding / Deocoding Video ways to encode/decode a video SW encoder/decoder Hybrid encoder/decoder HW (fixed-function) encoder/decoder SW Encoder Uses CPU Available almost immediately, at first usually very slow Uses instruction sets - SSE, AVX, NEON... and multi-threading if possible better efficiency than HW encoder not suitable for real-time encoding not suitable for devices with battery, slow CPUs Hybrid Solution CUVID HEVC on Intel Broadwell halfway there rsolution a way how to revive an older device HW Encoder/Decoder sometimes called fixed-function Digital Video 25  Uses ASIC - Application-specific integrated circuit integratedd circuit customized for a particular use, rather than intended for general-purpose use available approx. after 2 years since bitstream freeze to get better quality encodes, new HW necessary Intel Quick Sync Nvidia PureVideo HD AMD Video Core Next En embedded devices - TV, mobile SW Interfaces - APIS Windows - DXVA DXVA it allows video decoding to be hardaware-accelared Linux - NVENC/NVDEC, VA-API,V4L2 NVENC - Nvidia Encoder feature in Nvidia graphics cards it performs video encoding, offloading this compute- intensive task from the CPU to a dedicated part of the GPU macOS - VDA Support FFmpeg and anything which uses it VLC 3.0 MPC-HC on Windows Media Player Classic Adobe Encoder Software encode - CPU+CUDA/OpenCL OpenCL = a framework for writing programs that execute across heterogeneous platforms consisting of CPUs, GPUs, Digital Video 26  digital signal processors, field-programmable gate arrays and other processors or hardware accelerators Hardware encode - CPU+CUDA/OpenCL+QuickSync/NVENC CPU - not accelerated effects GPU - accelerated effects - CUDA/OpenCL Scaling (HD to SD; SD to HD) Timecode filter, Deinterlacing Pixel format conversion Change of aspect ration everything in Efects Quick Sync - fast encoding video, bad quality, not suitable for final_final_really_final last_3_v2 SDI serial digital interface used in broadcasting environment coaxial cable with BNC conncector the only way how to get camera output to computer (HDMI) up to 10-bit, 4:2:2 can transfer also audio and timecode Digital Video 27  only single direction uses very expensive and complicated matrices Digital Video 28  SMPTE ST 2110 STANDARDS Set of standards defining transmission of AV over IP set of cooperating standards to label individual frames of video or film witha timecode timecodes are added to film, video, audio material timecode = a sequence of numeric codes generated at regular intervals by a timing synchronization system they provide a time reference for editing, synchonization and identification a form of media metadata >10 GbE networks with sound, metadata, timecode etc. Requires PTP very precise time synchronization agnostic to chrome subsampling, bit-depth or framerate More info NDI, NDI HX, HX2, HX3 Digital Video 29  Network Device Interface a royalty-free software specificatin to enable video- compatible products to communicate, deliver and receive high- definition video over a computer network in a high-quality, low-latency manner that is frame accurate and suitable for switching in a live production environment Created by NewTek AV over ∼1 GbE IP Royalty-free license but not open Poor man's SMPTE ST 2110 Uses lossy compression algorithms with audio, tally supported in Unreal Engine HX and HX2 use H.264, H.265 resp. Video Distribution Digital Video 30  DVD-Video Digital Versatile Disc, not Video Disc successor of VCD and SVCD Single Layer - 4.7GB, Dual Layer 8.5GB Double-sided - 17.08GB standard DVD Book Part 3, managed by DVD Forum 720×576(D1), interlaced, MPEG-2, up to 9.8Mb/s PCM, AC-3, DTS, MP2, 48-96 kHz - depends on # of channels Subtitles (as images), images, menu, chapters, limited interactivity supports anamorphic widescreen video Locked to 9 regions Blu-Ray Disc and Ultra HD Blu-Ray named after blue laser it uses - 100GB TL standard, managed by Blu-ray Disc Association MPEG-2, VC-1, H.264 and UHD also H.265 3D video - fail, therefore not in UHD PCM, AC-3, DTS, DRA, 48-192 kHz - depends on # of channels Dolby Atmos and DTS:X Subtitles, images, menu, chapters, JAVA History of TV 2nd half 19th century - 30's - mechanical TV - approx. 100 lines end of 19th century - 70's - grayscale analog TV - 625 lines 50's - late 2010's - colour analog TV PAL, NTSC and SECAM systems - in CSR until 1993 SECAM, then PAL end of 70's - early 90's - colour analog HDTV - Japan 90's in US - EDTV - Enhanced-definition TV Digital Video 31  end of 80's - now - digital SDTV 90's - now - digital HDTV 00's - now - digital UHDTV-1 2010's - distant future (especially for CZ) - UHDTV-2 Broadcast TV Distribution Terrestrial Satellite - Skylink, international Cable - UPC, mainly in USA IPTV - O2TV Internet - Youtube, iVysílání, Skylink Live Digital TV Standards 4 main standards in whole world DVB - everybody except bellow ATSC - USA, Canada, Mexico and South Korea ISDB - Japan, Philippines, South America, Botswana DTMB - China, Cuba, other DVB DVB Project > 270 members European project of ETSI, CENELEC and EBU Standard does not specify codecs, but physical layer - signal distribution DVB-T - 32 Mb/s DVB-T2 - 50 Mb/s DVB-S - 55 Mb/s DVB-S2 - 146 Mb/s DVB-S2x - 191 Mb/s DVB-C - 64 Mb/s DVB-C2 - 83 Mb/s Digital Video 32  Mobile - H, NGH, SH - fail Properties of DVB-T2 Broadcast multiplexing - multiple channels in single stream in CZ e.g. Multiplex 21 - channels of Czech TV HbbTV - hybrid TV - internet services in TV, e.g. iVysílání Teletext over HbbTV - news, ads, closed captions EPG electronic program guide surround sound, multiple audio tracks, closed captions includes radio channels DVB-T in CZ DVB-T - switch began in 2005, broadcast ends 2020 DVB-T2 - switch began 2017 switch has to be done to use radio spectrum more efficiently better service, higher quality, more channels specifications are in D-Book by ČTÚ UHD supports HDR10, optional HDR10+ and Dolby Vision, 10- bit, Rec. 2020 Streaming YouTube, Netflix, iVysílání, DVTV... protocols for streaming over HTTP to viewer: MPEG Dynamic Adaptive Streaming over HTTP (DASH) - open standard Apple's HTTP Live Streaming (HLS) - limited codec support usually transports H.264, VP9, AV1 with AAC, Opus or AC-3 protocols use for live-streaming to VOD service: DASH HLS Digital Video 33  Real-Time Messaging Protocol (RTMPS) usually requires a key to identify the streamer usually in H.264 or H.265 with AAC real-time transcoding necessary Digital Cinema DCI - Digital Cinema Initiatives - since 2002 DCP - Digital Cinema Package - Digital Cinema System Specification v1.4.1 2K - 2048×10802048×1080 - 24 and 48fps 4K - 4096×21604096×2160 - 24 fps 12-bit JPEG 2000 up to 250 Mb/s, 96 or 48 kHz 24-bit PCM - MXF files Uses XML for metadata supported in Adobe, DCP-o-matic (FLOSS) FFMPEG https://jaroslavsvoboda.eu/vyuka/multitech/ffmpeg.html#/title- slide free and open source software for multimedia encoding, decoding and editing Started in 2000, written in C/C++ Current version is 5.0.1 (we will use 4.2.4) Used by Blender, Chrome, mpv, VLC, OBS Used by YouTube and BiliBili MediaCMS, Open Streaming Platform, Owncast, PeerTube, LBRY Used by many smartphone applications Framework with "plugins" can be used with multiple SW and HW codecs and devices - webcam, video capture card... Digital Video 34  Libraries for video, audio, subtitles, images, devices, streaming protocols, encryption... You can use it for: streaming, encoding, transcoding, transmuxing, recording, decoding, editing, filtering... transcoding process input file → demuxer → encoded data packets → decoder → decoded frames → encoder → encoded data packets → muxer → output file Simple default conversion: ffmpeg -i {input} {output} Specify codecs: -c:v {vcodec} -c:a {acodec} Scale the video: -s {horz}x{vert} Specify rates: -r:a {audio rate} -b:v {video bitrate} -b:a {audio bitrate} Or even better, use CRF with 26x: -crf {crf rate} Change duration: -ss {hrs}:{mins}:{secs}.{msecs} -t {hrs}:{mins}:{secs}.{msecs} Fun audio + image mix: ffmpeg -i {audio} -i {image} {output} LAWS OF ENCODING Digital Video 35  You can have fast, high-quality encoding, but the file will be huge You can have high-quality, small file, but the encoding will take forever You can have small files with fast encoding, but the quality will be horrible LIBRARIES FOR AUDIO ENCODING libmp3lame, libshine MP3 libfdk_aac, aac AAC-LC a HE-AAC(not aac) libopus, opus Opus wavpack, libwavpack WavPack ac3 Dolby Digital flac FLAC libvorbis Vorbis - OGG libopencore-amrnb, libvo-amrwbenc, libtwolame AMR-NB, AMR-WB, MP2 order from best to worst quality libopus > libvorbis ≥ libfdk_aac > aac > libmp3lame ≥ eac3/ac3 > libtwolame > vorbis > mp2 > wmav2/wmav1 Elementary Syntax Digital Video 36  3 binaries ffmpeg processes multimedia files encoding, transcoding, filtering etc. ffplay player ffprobe multimedia files info complement to mediainfo [binary] [input] [encoding presets] [output] show help - [binary] -h show licence - [binary] -L list formats - [binary] -formats list demuxers - [binary] -demuxers list muxers - [binary] -demuxers list devices - [binary] -devices list codecs - [binary] -codecs list decoders - [binary] -decoders list encoders - [binary] -encoders list protocols - [binary] -protocols list filters - [binary] -filters list pixel formats - [binary] -pix_fmts list sampling formats - [binary] -sample_fmts list audio channels - [binary] -layouts list colors - [binary] -colors hide banner - [binary] -hide_banner Digital Video 37