2-ImageCharacteristics+-+Students copy.pptx

CSC3067 Video Analytics and Machine Learning Week 1 - Image and video acquisition Image Processing Video Processing Histograms Summary of last lecture Module Overview 1. Introduction to Video-Surveillance and Computer Vision PAR...

CSC3067 Video Analytics and Machine Learning Week 1 - Image and video acquisition Image Processing Video Processing Histograms Summary of last lecture Module Overview 1. Introduction to Video-Surveillance and Computer Vision PART 1: IMAGE AND VIDEO PROCESSING 2. Image and video acquisition and characteristics 3. Data Preprocessing: Point Operations Brightness enhancement Contrast enhancement 4. Data Preprocessing: Neighbourhood Operations Filtering and Noise reduction. Convolution 5. Image Segmentation Brightness segmentation Template Matching 6. Video Segmentation: Motion Estimation Background Subtraction Background Mixture Models, Optical Flow 7. Video Segmentation: Tracking Kalman Filter Particle Filter Tracking by Detection Paintings Print media Photographs Television Computer screen IMAGE REPRESENTATION  Digital Image Acquisition Source of imagery the ubiquitous video camera Charge-coupled device (CCD) sensor 3CCD for colour images Rectangular array of detectors Measure brightness (intensity) and colour Digital Image Representation Optical image denoted by I(x,y) x and y are spatial co-ordinates I(x,y) is the brightness at position (x,y) CCD sensor Image y I(x,y) lens x  Digital Representation M=128 N=128 picture element (pixel) Digital Representation “Real World” Digital Capture I(x,y) Ii,j x j column y i Sampling interval ,∆ 𝑥 row Continuous Discrete Image can be understood as a bi- dimensional array with 2 indexes char [][] image = new char Image Characteristics Spatial Resolution Dynamic Range Gray-level quantisation Colour Space  Image Resolution Measure of image quality in terms of detail Given by the CCD sensor Technological limit Number of pixels in image array = M × N (no. of rows by no. of columns) 4500 x 3000 digital photo camera 1080 x 920 HD video camera 720 x 480 video camera 256 x 256 reasonable processing quality  Dynamic Range A measure of the range of brightness values that can be detected by a sensor. DR = Largest amount of light – Smallest amount of light Normal imagery: pixel is typically in the range 0 - 255 (8 bits) - 256 shades of grey (‘grey levels’) - 0 for black, 255 for white Medical imagery: 0 - 4095 = 12 bits SLR cameras (Raw images): 12-14 bits Astronomical imagery: 0 - 65535 = 16 bits Incorrect use of the dynamic range will created saturated/under exposed images  Dynamic Range (a) Under-exposed image, (b) normally-exposed image, (c) over-exposed image  Grey-level quantisation A measure of how accurate the digitisation 256 levels 11 levels 5 levels process is. For a given DR, quantisation represents how many levels of gray we use For display, normally 8 bits per pixel (bpp) is used. The human eye cannot resolve to this accuracy. It looks a continuous gradient for us 32 gray levels are usually sufficient (5 bpp). At 4 bpp and below, “false contouring” can become apparent.  Colour A gray-scale image is a matrix containing the brightness value I for each pixel (i,j) Grayscale images have one channel A colour image is composed by several channels (matrices), each containing the brightness in each channel Channels represent primary colours Combinations of primary colours produce the colour spectrum or colour space Colour Image can be understood as a three-dimensional array with 3 indexes char [][][] image = new char Colour Spaces A colour space a is mathematical model where the colours can be represented as a vector of numbers, typically as three or four values or colour components Each colour space has certain features that can be beneficial for characterising an object Invariance to colour variation, shadows, reflections, illumination changes RGB is the most common one It is represented as a cube HSV is the most invariant one Cylindrical/conical space Hue (colour), saturation (intensity), value (brightness) Colour can be defined as hue + saturation By removing value our feature is invariant to illumination changes! And many other spaces: CMYK, HSL, YCbCr  N Grey levels IMAGE HISTOGRAMS Image Histograms N Grey levels Shows the distribution of image pixels in terms of their grey levels. H i   N i Plot of Ni versus i Grey levels i=0,1,…,255 char [] hist = new char Ni= number of pixels in image with grey level i Cumulative Histogram H(J) = number of pixels with value == J G CH(G) = number of pixels with value Slow motion effect Low framerates (1-2 fps) for surveillance applications Saving space in video recordings  Compression Goal: Reduce the amount of information (bits) to be transmitted/stored By taking advantage of redundancy in images and videos Temporal redundancy Spatial Information (also for images) 8bits 4 bits 124 126 122 125 124 +2 -4 +3 113 111 121 122 -11 -2 +10 +1 114 126 120 116 +1 +12 -6 -4 100 103 115 115 -14 +2 +12 0 8bits x 4x4 = 128 bits 8bits +4bits x (4x4-1) = 68 bits  Compression Temporal redundancy 124 126 122 125125 124 118 125 124 126 122 125+1 +2 +4 0 113 111 121 122113 111 121 122 113 111 121 1220 0 0 0 114 126 120 116116 120 119 109 114 126 120 116+2 -6 -1 -5 time 100 103 115 115101 102 115 114 100 103 115 115+1 -1 0 -1 Keyframes: From time to time you expect a big change (change of scene, sudden movement) where temporal compression does not implies any advantage, so you send a frame without temporal compression. keyframes  Compression 2 types: Lossless compression Original data can be perfectly reconstructed from the compressed data Lossy compression Better compression rates Reconstruction only of an approximation of the original data  Compression Most popular: M-Jpeg, Mpeg, H.264, DIVX, WMV, Sorenson, 3GP Do not confuse with containers: AVI, MOV Containers are file formats, can use any of previous codecs Also in images: JPEG, GIF, PNG What did we cover today? Image representation Resolution, dynamic range, quantisation, colour Histograms Video representation Frame rate, compression