Lecture 6: Fundamentals of Digital Video PDF

Lecture 6: Fundamentals of Digital Video Copyright © 2016 Pearson Education, Inc. All Rights Reserved  Learning Objectives 6.1 The common terms in digital video. 6.2 The common terms for DTV. 6.3 The file formats of high-definition and standard-definition digital video. 6.4 The...

Lecture 6: Fundamentals of Digital Video Copyright © 2016 Pearson Education, Inc. All Rights Reserved  Learning Objectives 6.1 The common terms in digital video. 6.2 The common terms for DTV. 6.3 The file formats of high-definition and standard-definition digital video. 6.4 The relationship among frame size, frame aspect ratio, and pixel aspect ratio. 6.5 How pixel aspect ratio affects the appearance of the video picture. Copyright © 2016 Pearson Education, Inc. All Rights Reserved  Learning Objectives 6.6 How to read video timecodes. 6,7 The different implications of data rate and file size. 6.8 How to determine the suitable video data rate for a playback target. 6.9 The general strategies for video file size optimization. Copyright © 2016 Pearson Education, Inc. All Rights Reserved Video Frame Rate – How fast the pictures are captured – How fast the frames are played back is determined by – Frames per second (fps) Copyright © 2016 Pearson Education, Inc. All Rights Reserved  Standards for Analog Color TV NTSC: – National Television Systems Committee (US) – U.S., Japan, Taiwan, parts of the Carribean, South America – Frame Rate: 30 fps( Black and White)/ 29.97 fps (Color) PAL: – Phase Alternating Line – Australia, New Zealand, Western Europe, Asian – Frame Rate: 25 fps SECAM: – Séquentiel Couleur avec Mémoire – France, former Soviet Union, Eastern Europe – Frame Rate: 25 fps Motion Picture Film frame rate is 24 fps Copyright © 2016 Pearson Education, Inc. All Rights Reserved How CRT Monitors and TVs Display Pictures Picture displayed on CRT is made up of horizontal lines – NTSC: 525 lines (about 480 lines are picture) – PAL and SECAM: 625 lines (about 576 lines are picture) Lines are traced across the screen – one line at a time – from top to bottom Progressive scan: – from top to bottom in one pass Interlaced scan: – in two passes: 1. even-numbered lines 2. odd-numbered lines Copyright © 2016 Pearson Education, Inc. All Rights Reserved  Undesirable Side Effects of Interlaced Scan Field : Set of lines in the same pass – 2 fields in interlaced scan. Even and Odd numbered lines – Upper field: field that contains the topmost scan lines – Lower field: the other field The two fields in a frame are captured at a slightly different moment in time Discontinuities will become apparent for fast moving objects in video shot in the interlaced mode Comb-like artifacts / Interlace Artifacts Copyright © 2016 Pearson Education, Inc. All Rights Reserved Comb-like Artifacts / Interlace Artifacts Not discernible during normal playback of most videos Deinterlace: To remove the interlace artifact Common method: – discard one field – fill in the gaps by duplicating or interpolating the other field Copyright © 2016 Pearson Education, Inc. All Rights Reserved Color Format for Videos Luminance-chrominance color models Luminance: brightness Chrominance: color or hue YUV: – Y: luminance component – U and V: chrominance components – used for PAL YIQ: – Y: luminance component – I and Q: chrominance components – used for NTSC Copyright © 2016 Pearson Education, Inc. All Rights Reserved Luminance-Chrominance Color Model Examples YUV: – Y: luminance component – U and V: chrominance components – used for PAL YIQ: – Y: luminance component – I and Q: chrominance components – used for NTSC Copyright © 2016 Pearson Education, Inc. All Rights Reserved  Sampling and Quantization of Motion (1 of 2) Temporal: – sampling rate: ▪ how frequent you take a snapshot of the motion ▪ Frame digitized based on how image is digitized ▪ frame rate: Number of frames per second ▪ higher sampling rate: higher frame rate ▪ higher frame rate  more frames for the same duration  larger file size Frame Size : Resolution of the frame. – Measured in pixel dimension – No ppi setting because not intended for print but display Copyright © 2016 Pearson Education, Inc. All Rights Reserved Frame Size Examples / Frame Aspect Ratio Blank Blank Frame size NTSC standard definition 720  480 pixels Blank high definition 1280  720 pixels PAL standard definition 720  576 pixels the ratio of a frame’s viewing width to height NOT equivalent to ratio of the frame’s pixel width to height. Standard definition NTSC wide-screen format High definition digital video High definition TV Copyright © 2016 Pearson Education, Inc. All Rights Reserved Ratio Does Not Match Up? Frame size of a NTSC standard definition DV frame:720  480 720 : 480 = 3 : 2 NOT 4 : 3 or 16 : 9 This is because the pixels are not square! Digital images: square pixels Digital video: may not be square pixels Ratio of pixel width : pixelheight Pixel Aspect Ratio Pixel Shape 1 square 1 wide Copyright © 2016, 2011, 2007 Pearson Education, Inc. All Rights Reserved Pixel Aspect Ratio Examples Video Format Pixel Aspect Ratio Standard format of standard definition 0.9 (e.g. standard format of the non-blu- ray movies DVD) Wide-screen format of standard 1.2 definition (e.g. wide-screen format of the non-blu-ray movies DVD) 720p, 720i, 1080p, 1080i, QuickTime 1.0 movies Copyright © 2016 Pearson Education, Inc. All Rights Reserved Standard Definition Copyright © 2016 Pearson Education, Inc. All Rights Reserved Distortion Video Image Will Be Distorted If It Is Displayed On A System With A Different Pixel Aspect Ratio Pixel Apect Ratios Distortion video frame’s = display system’s none video frame’s < display system’s stretched horizontally video frame's > display system’s stretched vertically Copyright © 2016 Pearson Education, Inc. All Rights Reserved Counting Time in Digital Video Timecode: to number frames SMPTE (Society of Motion Pictures and Television Engineers) video timecode – number frames in hours, minutes, seconds, and frames – drop-frame timecode – non-drop-frame timecode Non-Drop-Frame Timecode – Example: 00:02:51:20 Video time is 0 hours, 2 minutes, 51 seconds, and 20 frames – Use colons Copyright © 2016 Pearson Education, Inc. All Rights Reserved Drop-Frame Timecode (1 of 2) Example: 00;02;51;20 Video time is 0 hours, 2 minutes, 51 seconds, and 20 frames Use semi-colons (not colons) Preferable for the NTSC system Does NOT mean dropping or removing frames from the video NTSC video is 29.97 fps, not exactly 30 fps Drop-frame timecode renumbers frames to make the timecode more accurately represent the video time based on 29.97 fps Copyright © 2016 Pearson Education, Inc. All Rights Reserved Digital Video (DV) Digital Video Standards – Standard definition – High definition – Digital Television DV compression and DV format: specific types of digital video compression and format respectively In this lecture – DV refers to the specific types of digital video Copyright © 2016 Pearson Education, Inc. All Rights Reserved Standard Definition DV25 Format (1 of 3) Copyright © 2016 Pearson Education, Inc. All Rights Reserved Advantages Using YUV over RGB A luminance-chrominance color model YUV – Y: luminance (brightness) component U: a chrominance (color or hue) component V : a chrominance (color or hue) component The human eye is more sensitive to changes of the luminance than it is to chrominance changes. Thus, with YUV, we can assign fewer bits to store the chrominance components. Chroma subsampling or color subsampling Fewer bits means smaller file size. Copyright © 2016 Pearson Education, Inc. All Rights Reserved The Three Numbers in YUV Ratio of numbers of samples of Y : U : V for each group of 4 pixels YUV No. of No. of No. of Total no. Saving in storage 4:4:4 samples of Y samples of U samples of YUV for each for each group of V for samples group of 4 of 4 pixels each of for 4:2:2 pixels group of each 4 pixels group of 4:2:0 4 pixels 4:4:4 4 4 4 4+4+4=12 no compression 4:1:1 4:2:2 4 2 2 4+2+2=8 Reduced by 4 33% compression 4:2:0 4 2 0 4+2+0=6 Reduced by 6 0 2 50% compression 4:1:1 4 1 1 4+1+1=6 Reduced by 6 50% compression Copyright © 2016 Pearson Education, Inc. All Rights Reserved High Definition Common high definition video formats: – HDV – DVCPro HD – AVCHD – AVC-Intra Different video cameras support different high def. format The name of the supported format is printed on the body of the video camera Copyright © 2016 Pearson Education, Inc. All Rights Reserved Commalities Among High Definition Specifications (1 of 2) Frame aspect ratio: 16:9 Picture formats: – 1080 and/or 720 – interlace and/or progressive For 1080, possible frame sizes: – 1920  1080 (pixel aspect ratio: 1.0) – 1440  1080 (pixel aspect ratio: 1.333) - older format For 720, possible frame sizes: – 1280 x 720 (pixel aspect ratio: 1.0) – 960 x 720 (pixel aspect ratio: 1.333) – older format Color sampling methods: 4:2:0 or 4: 2: 2 Copyright © 2016, 2011, 2007 Pearson Education, Inc. All Rights Reserved Picture Format Notation Copyright © 2016 Pearson Education, Inc. All Rights Reserved Digital Television (DTV) (2 of 2) Signals of DTV are broadcast and transmitted digitally Need a digital TV set to watch Standard definition – 704  480, 16 : 9 and 4 : 3, progressive and interlaced – 640  480, 4 : 3, progressive and interlaced High definition – 1920  1080, 16 : 9, progressive and interlaced – 1280  720, 16 : 9, progressive and interlaced MPEG-2 Copyright © 2016, 2011, 2007 Pearson Education, Inc. All Rights Reserved Common Video File Types.mov.mpg/.mpeg.divx.webm.fd4.mp4.avi.flv.ogg /.ogv.wmv Copyright © 2016 Pearson Education, Inc. All Rights Reserved Considerations for File Type File size restriction Intended audience – For Web: – Multiple platforms ▪ high compression ▪ Cross-platform ▪ streaming video formats: MPEG – DVD/Blu-ray video: – How audience will watch the video ▪ MPEG-4/MPEG-2 Future editing – Lower compression level Copyright © 2016 Pearson Education, Inc. All Rights Reserved Digital Video File Size Optimization Video tends to have very large file size compared to other media. Why should we care file size optimization? – A large file requires more disk space. – A large file takes longer to transfer. – Data transfer can be expensive (because data plans are not unlimited) – High data rate may cause choppy playback of the video. (Data rate will be explained later in this lecture.) Copyright © 2016 Pearson Education, Inc. All Rights Reserved  Uncompressed 1-Second Video File Size 1920 × 1080 pixels ; 24−bit color = 1,492,992,000 bits/ (8 bits/bytes) 30 fps ; 1 second long = 186,624,000 bytes Audio: stereo (2 channels)  178 MB Audio: 48 kHz, 16-bit The picture component: Total pixels in each frame: 1920 1080 Pixels = 2,073,600 pixels/frames File size of each frame: 2,073,600 pixels/frames  24 bits/ Pixels = 49,766,400 bits/frame File size of 30 frames (1 second): 49,766,400 bits/frame  30 frames = 1,492,992,000 bits Copyright © 2016 Pearson Education, Inc. All Rights Reserved  File Size Calculation (3 of 5) Audio : Sampling rate  length of the audio  bit-depth  number of channels = 48,000 samples/sec 1 second 16 bits/sample  2 = 1,536,000 bits = 1,536,000 bits/(8 bits/byte) = 192,000  188 KB bytes 10 seconds would be 1.65 G B! Total file size of this 1-second uncompressed video = size of the picture component + audio size = 178 MB + 188 KB  178 MB Data Rate : Amount of video data to be processed per second File Size Average Data Rate = Duration of Video (seconds) Copyright © 2016 Pearson Education, Inc. All Rights Reserved Effect of File Size Versus Data Rate on Video Playback Data rate: – If high: choppy playback – Amount of data to be processed per second ▪ Larger file size can have a low data rate if it is a long video ▪ Smaller file size can have a high data rate if it is a short video File size: – If high: ▪ Requires larger storage space ▪ Not unnecessary choppy playback – The impact of file size on smoothness of playback also depends on the video duration. Copyright © 2016 Pearson Education, Inc. All Rights Reserved Data Rate Example (1 of 2) Average download speed of 3G wireless: about 1 mbits/sec Average download speed of 4G wireless: 5 − 12 mbits/sec Previous video file size example: 1-second uncompression video, 178 MB − Data rate = 178 MB/1second = 178 MB/sec = 1,424 mbits/sec − Way too high for any of these connections! − Impossible to play back smoothly via these connection Copyright © 2016 Pearson Education, Inc. All Rights Reserved General Strategies for Reducing Video File Size General Strategies for reducing digital image file size – reduce frame size – reduce frame rate – choose a video compressor that allows higher compression, example H.264 – choose the lower picture quality option Reduce duration of the video so you have less frames – not always possible – will not impact data rate Copyright © 2016 Pearson Education, Inc. All Rights Reserved Compression Basic idea: – Want to represent the same content by using less data – Codec: Compressor / decompressor Compression: – To reduce file size – Takes time – Often takes more time for higher compression Decompression: – A compression video file must be decompressed before it is played. – The decompression method or algorithm depends on how it is originally compressed. Copyright © 2016 Pearson Education, Inc. All Rights Reserved Spatial Compression (1 of 2) Compact individual frames as if they are independent digital images. Good for cartoon animation videos Examples of algorithms: – Run-length encoding (RLE) – JPEG compression Example codecs: – QuickTime Animation – QuickTime PlanarRGB – Microsoft RLE – Disadvantage: Less compressed Copyright © 2016 Pearson Education, Inc. All Rights Reserved Temporal Compression (1 of 2) Exploits the repetitious nature of image content over time in video. Good for continuous motion videos Saving more information for selected frames, i.e. less compressed. These are called key frames. All other frames stores only the difference from the previous key frame, instead of full frame Advantage: – Effective if the change between a frame and its previous key frame is small Example Codec: H.264, Sorenson Video Copyright © 2016 Pearson Education, Inc. All Rights Reserved Lossy versus Lossless Compression Lossy compression: – Reduce data by discarding or altering some of the original data. Low picture quality – Much smaller file size than lossless compression Lossless compression: – Preserve the original data but reduce file size by encoding the data specially – Example codecs: ▪ QuickTime Animation ▪ PlanarRGB Copyright © 2016 Pearson Education, Inc. All Rights Reserved Symmetrical and Asymmetrical Compression Symmetrical codec: – Same amount of time in compression and decompression Asymmetrical codec: – Amount of time to compress and decompress are significantly different – Preferable: Fast decompression so less wait time to play back the video Copyright © 2016 Pearson Education, Inc. All Rights Reserved