Computer Vision Lecture Notes PDF

Computer Vision By Dr. Ahmed Taha Lecturer, Computer Science Department, Faculty of Computers & Artificial Intelligence, Benha University 1  Resizing, Filtering, and Convolutions Lecture Four...

Computer Vision By Dr. Ahmed Taha Lecturer, Computer Science Department, Faculty of Computers & Artificial Intelligence, Benha University 1  Resizing, Filtering, and Convolutions Lecture Four 2 3 4 Interpolation interpolation - the insertion of an intermediate value or term into a series by estimating or calculating it from surrounding known values Triangle 5 Image Resolution Resolution refers to the number of pixels in an image. Resolution is sometimes identified by the width and height of the image as well as the total number of pixels in the image. For example, an image that is 2048 pixels wide and 1536 pixels high (2048 x 1536) contains 1536 (multiply) 3,145,728 pixels (or 3.1 Megapixels). 2048 6 Image Resolution 7 Image Resolution 8 Print Resolution Print Resolution refers to the number of dots per inch (dpi). 9 10 Image resizing! Say we want to increase the size of an image… This is a beautiful image of a sunset (4×4)… it’s just very small… 11 Image resizing! Say we want to increase the size of an image… This is a beautiful image of a sunset… it’s just very small… Say we want to increase size 4x4 -> 7x7 12 Resize 4x4 -> 7x7 (Enlarging) - Create our new image 13 Resize 4x4 -> 7x7 - Create our new image - Match up coordinates 14 Resize 4x4 -> 7x7 - Create our new image - Match up coordinates 15 Resize 4x4 -> 7x7 - Create our new image (-0.5, -0.5) - Match up coordinates (3.5, 3.5) 16 Resize 4x4 -> 7x7 - Create our new image - Match up coordinates - System of equations - aX + b = Y - a*-0.5 + b = -0.5 - a*6.5 + b = 3.5 - From a*-0.5 + b = -0.5 - b=-0.5+0.5a - Substitute in a*6.5 + b = 3.5 - a*6.5-0.5+0.5*a =3.5 - 7*a=4 17 Resize 4x4 -> 7x7 - Create our new image - Match up coordinates - System of equations - aX + b = Y - a*-0.5 + b = -0.5 - a*6.5 + b = 3.5 - a*7 = 4 18 Resize 4x4 -> 7x7 - Create our new image - Match up coordinates - System of equations - aX + b = Y - a*-0.5 + b = -0.5 - a*6.5 + b = 3.5 - a*7 = 4 - a = 4/7 19 Resize 4x4 -> 7x7 - Create our new image - Match up coordinates - System of equations - aX + b = Y - a*-0.5 + b = -0.5 - a*6.5 + b = 3.5 - a = 4/7 20 Resize 4x4 -> 7x7 - Create our new image - Match up coordinates - System of equations - aX + b = Y - a*-0.5 + b = -0.5 - a*6.5 + b = 3.5 - a = 4/7 - a*-0.5 + b = -0.5 21 Resize 4x4 -> 7x7 - Create our new image - Match up coordinates - System of equations - aX + b = Y - a*-0.5 + b = -0.5 - a*6.5 + b = 3.5 - a = 4/7 - a*-0.5 + b = -0.5 - 4/7*-1/2 + b = -1/2 22 Resize 4x4 -> 7x7 - Create our new image - Match up coordinates - System of equations - aX + b = Y - a*-0.5 + b = -0.5 - a*6.5 + b = 3.5 - a = 4/7 - a*-0.5 + b = -0.5 - 4/7*-1/2 + b = -1/2 - -4/14 + b = -7/14 23 Resize 4x4 -> 7x7 - Create our new image - Match up coordinates - System of equations - aX + b = Y - a*-0.5 + b = -0.5 - a*6.5 + b = 3.5 - a = 4/7 - a*-0.5 + b = -0.5 - 4/7*-1/2 + b = -1/2 - -4/14 + b = -7/14 - b = -3/14 24 Resize 4x4 -> 7x7 - Create our new image - Match up coordinates - System of equations - aX + b = Y - a*-0.5 + b = -0.5 - a*6.5 + b = 3.5 - a = 4/7 - b = -3/14 25 Resize 4x4 -> 7x7 - Create our new image - Match up coordinates - 4/7 X - 3/14 = Y 26 Resize 4x4 -> 7x7 - Create our new image - Match up coordinates - 4/7 X - 3/14 = Y 27 Resize 4x4 -> 7x7 - Create our new image - Match up coordinates - 4/7 X - 3/14 = Y - Iterate over new pts 28 Resize 4x4 -> 7x7 - Create our new image - Match up coordinates - 4/7 X - 3/14 = Y - Iterate over new pts - Map to old coords 29 Resize 4x4 -> 7x7 - Create our new image - Match up coordinates - 4/7 X - 3/14 = Y - Iterate over new pts - Map to old coords - (1, 3) 30 Resize 4x4 -> 7x7 - Create our new image - Match up coordinates - 4/7 X - 3/14 = Y - Iterate over new pts - Map to old coords - (1, 3) - 4/7*1 - 3/14 - 4/7*3 - 3/14 31 Resize 4x4 -> 7x7 - Create our new image - Match up coordinates - 4/7 X - 3/14 = Y - Iterate over new pts - Map to old coords - (1, 3) - 4/7*1 - 3/14 - 4/7*3 - 3/14 - (5/14, 21/14) ≈ (0.36,1.5) 32 Resize 4x4 -> 7x7 - Create our new image - Match up coordinates - 4/7 X - 3/14 = Y - Iterate over new pts - Map to old coords - (1, 3) -> (5/14, 21/14) 33 Resize 4x4 -> 7x7 - Create our new image - Match up coordinates - 4/7 X - 3/14 = Y - Iterate over new pts - Map to old coords - (1, 3) -> (5/14, 21/14) - Interpolate old values 34 Resize 4x4 -> 7x7 - Create our new image - Match up coordinates - 4/7 X - 3/14 = Y - Iterate over new pts - Map to old coords - (1, 3) -> (5/14, 21/14) - Interpolate old values 35 Resize 4x4 -> 7x7 - Create our new image - Match up coordinates - 4/7 X - 3/14 = Y - Iterate over new pts - Map to old coords - (1, 3) -> (5/14, 21/14) - Interpolate old values - Size of opposite rects 36 Resize 4x4 -> 7x7 - Create our new image - Match up coordinates - 4/7 X - 3/14 = Y - Iterate over new pts - Map to old coords - (1, 3) -> (5/14, 21/14) - Interpolate old values - Size of opposite rects - OR find q1 and q2, then interpolate between them 37 Resize 4x4 -> 7x7 - Create our new image - Match up coordinates - 4/7 X - 3/14 = Y - Iterate over new pts - Map to old coords - (1, 3) -> (5/14, 21/14) - Interpolate old values - q1 = r1, g1, b1 - r1 = 0.5*0 + 0.5*241 - g1 = 0.5*255 + 0.5*90 - b1 = 0.5*255 + 0.5*36 38 Resize 4x4 -> 7x7 - Create our new image - Match up coordinates - 4/7 X - 3/14 = Y - Iterate over new pts - Map to old coords - (1, 3) -> (5/14, 21/14) - Interpolate old values - q1 = (120.5, 172.5, 145.5) - q2 = r2, g2, b2 39 Resize 4x4 -> 7x7 - Create our new image - Match up coordinates - 4/7 X - 3/14 = Y - Iterate over new pts - Map to old coords - (1, 3) -> (5/14, 21/14) - Interpolate old values - q1 = (120.5, 172.5, 145.5) - q2 = r2, g2, b2 - r2 = 0.5*241 + 0.5*255 - g2 = 0.5*90 + 0.5*255 - b2 = 0.5*36 + 0.5*0 40 Resize 4x4 -> 7x7 - Create our new image - Match up coordinates - 4/7 X - 3/14 = Y - Iterate over new pts - Map to old coords - (1, 3) -> (5/14, 21/14) - Interpolate old values - q1 = (120.5, 172.5, 145.5) - q2 = (248, 172.5, 18) 41 Resize 4x4 -> 7x7 - Create our new image - Match up coordinates - 4/7 X - 3/14 = Y - Iterate over new pts - Map to old coords - (1, 3) -> (5/14, 21/14) - Interpolate old values - q1 = (120.5, 172.5, 145.5) - q2 = (248, 172.5, 18) - q = r, g, b - w = 9/14*w1 + 5/14 w2 42  Resize 4x4 -> 7x7 - Create our new image - Match up coordinates - 4/7 X - 3/14 = Y - Iterate over new pts - Map to old coords - (1, 3) -> (5/14, 21/14) - Interpolate old values - q1 = (120.5, 172.5, 145.5) - q2 = (248, 172.5, 18) - q = r, g, b - w = 9/14*w1 + 5/14 w2 - r = 9/14*120.5 + 5/14*248 - g = 9/14*172.5 + 5/14*172.5 43 - b = 9/14*145.5 + 5/14*18 Resize 4x4 -> 7x7 - Create our new image - Match up coordinates - 4/7 X - 3/14 = Y - Iterate over new pts - Map to old coords - (1, 3) -> (5/14, 21/14) - Interpolate old values - q1 = (120.5, 172.5, 145.5) - q2 = (248, 172.5, 18) - q = (166, 172.5, 100) 44 Resize 4x4 -> 7x7 - Create our new image - Match up coordinates - 4/7 X - 3/14 = Y - Iterate over new pts - Map to old coords - (1, 3) -> (5/14, 21/14) - Interpolate old values - q = (166, 172.5, 100) 45 Resize 4x4 -> 7x7 - Create our new image - Match up coordinates - 4/7 X - 3/14 = Y - Iterate over new pts - Map to old coords - (1, 3) -> (5/14, 21/14) - Interpolate old values - q = (166, 172.5, 100) 46 Resize 4x4 -> 7x7 - Create our new image - Match up coordinates - 4/7 X - 3/14 = Y - Iterate over new pts - Map to old coords - (1, 3) -> (5/14, 21/14) - Interpolate old values - q = (166, 172.5, 100) 47 Resize 4x4 -> 7x7 - Create our new image - Match up coordinates - 4/7 X - 3/14 = Y - Iterate over new pts - Map to old coords - (1, 3) -> (5/14, 21/14) - Interpolate old values - q = (166, 172.5, 100) - Fill in the rest 48 Resize 4x4 -> 7x7 - Create our new image - Match up coordinates - 4/7 X - 3/14 = Y - Iterate over new pts - Map to old coords - (1, 3) -> (5/14, 21/14) - Interpolate old values - q = (166, 172.5, 100) - Fill in the rest - On outer edges use padding! 49 Resize 4x4 -> 7x7 - Create our new image - Match up coordinates - 4/7 X - 3/14 = Y - Iterate over new pts - Map to old coords - (1, 3) -> (5/14, 21/14) - Interpolate old values - q = (166, 172.5, 100) - Fill in the rest 50 Resize 4x4 -> 7x7 - Create our new image - Match up coordinates - 4/7 X - 3/14 = Y - Iterate over new pts - Map to old coords - (1, 3) -> (5/14, 21/14) - Interpolate old values - q = (166, 172.5, 100) - Fill in the rest 51 We did it! 52 Different scales 4×4 256×256 Bilinear Interpolation 7×7 32×32 53 Different methods Cubic Interpolation 4×4 256×256 512×512 Bilinear Interpolation 7×7 32×32 Nearest Neighbor Interpolation 7×7 54 General steps for enlarging - Changing the interpolation method doesn’t change the overall method, so to enlarge an image we do the following: 1. Map the original and desired coordinates. 2. Find all the original (real) coordinates for each new coordinate. 3. Use an interpolation method to find each of the new pixel values. 55 56 Want to make image smaller 57 448x448 -> 64x64 (Shrinking) Image Thumbnail 58 448x448 -> 64x64 59 448x448 -> 64x64 Nearest-neighbor Interpolation Bilinear Interpolation 60 448x448 -> 64x64 61 448x448 -> 64x64 62 448x448 -> 64x64 63 448x448 -> 64x64 64 448x448 -> 64x64 65 448x448 -> 64x64 66 448x448 -> 64x64 67 448x448 -> 64x64 68 448x448 -> 64x64 69 448x448 -> 64x64 70 448x448 -> 64x64 71 448x448 -> 64x64 72 448x448 -> 64x64 73 448x448 -> 64x64 74 448x448 -> 64x64 75 448x448 -> 64x64 76 448x448 -> 64x64 77 448x448 -> 64x64 78 448x448 -> 64x64 79 448x448 -> 64x64 80 448x448 -> 64x64 81 448x448 -> 64x64 82 448x448 -> 64x64 83 448x448 -> 64x64 84 448x448 -> 64x64 85 448x448 -> 64x64 86 448x448 -> 64x64 87 448x448 -> 64x64 88 448x448 -> 64x64 89 448x448 -> 64x64 90 448x448 -> 64x64 91 448x448 -> 64x64 92 448x448 -> 64x64 93 448x448 -> 64x64 94 448x448 -> 64x64 95 448x448 -> 64x64 96 IS THIS ALL THERE IS?? 97 THERE IS A BETTER WAY! 98 LOOK AT HOW MUCH BETTER 99 How do? 100 How do? Averaging! 101 How do? Averaging! interpolation averaging 102 What is averaging? 103 What is averaging? A weighted sum 104 What is averaging? A weighted sum 105 Call this operation “convolution” Filter or kernel 106 Convolutions on larger images 107 Kernel slides across image 108 Convolutions on larger images 109 This is called box filter As the kernel is a rectangle, this is called a box filter. Box filters 110 Box filters smooth image Box filters 111 Box filters smooth image Box filters 112 Now we resize our smoothed image 113 So much better! 114 Box filters have artifacts 115 Box filters have artifacts 116 Box filters: vertical + horizontal streaking 117 Box filters: vertical + horizontal streaking 118 Box filters: vertical + horizontal streaking 119 Box filters: vertical + horizontal streaking 120 We want a smoothly weighted kernel 121 Gaussians 122 2d Gaussian 123 Better smoothing with Gaussians * = 124 Better smoothing with Gaussians Smoothing with Box filter Smoothing with Gaussian filter 125 Better smoothing with Gaussians Smoothing with Box filter Smoothing with Gaussian filter 126 Wow, so what was that convolution thing?? 127 Wow, so what was that convolution thing?? 128 Calculate it, go! 129 Calculate it, go! 130 Calculate it, go! 131 Guess that kernel! 132 Smoothing: Averaging 133 Guess that kernel! 134 Edges 135 Guess that kernel! 136 Identity Kernel: Does nothing! 137 Guess that kernel! 138 Sharpen Kernel: sharpens! 139 Guess that kernel! 140 Emboss Kernel: stylin’ 141 Guess those kernels! 142 Sobel Kernels: edges and… 143 Sobel Kernels: edges and gradient! 144 Sobel Kernels: edges and gradient! 145 And so much more!! 146 147