Computer Vision Lecture Notes PDF

Computer Vision By Dr. Ahmed Taha Lecturer, Computer Science Department, Faculty of Computers & Artificial Intelligence, Benha University 1  Edges and Features Lecture Five...

Computer Vision By Dr. Ahmed Taha Lecturer, Computer Science Department, Faculty of Computers & Artificial Intelligence, Benha University 1  Edges and Features Lecture Five 2 3 4 Convolution: Weighted sum over pixels 5 Filters 6 Convolution 7 Convolution 8 Convolution These are nicer mathematically 9 So what can we do with these convolutions anyway? Mathematically: all the nice things - Commutative - A*B = B*A - Associative - A*(B*C) = (A*B)*C - Distributes over addition - A*(B+C) = A*B + A*C - Plays well with scalars - x(A*B) = (xA)*B = A*(xB) 10 So what can we do with these convolutions anyway? This means some convolutions decompose: - 2d gaussian is just composition of 1d gaussians - Faster to run 2 1d convolutions 11 So what can we do with these convolutions anyway? - Blurring - Sharpening - Edges - Features - Derivatives - Super-resolution - Classification - Detection - Image captioning -... 12 So what can we do with these convolutions anyway? - Blurring All of computer vision - Sharpening is convolutions - Edges (basically) - Features - Derivatives - Super-resolution - Classification - Detection - Image captioning -... 13 So what can we do with these convolutions anyway? - Blurring All of computer vision - Sharpening is convolutions - Edges (basically) - Features - Derivatives - Super-resolution - Classification - Detection - Image captioning -... 14 What’s an edge? 2D and 3D representations of microscope images 15 What’s an edge? - Image is a function - Edges are rapid changes in this function 16 What’s an edge? - Image is a function - Edges are rapid changes in this function 17 Finding edges - Could take derivative - Edges = high response 18 Derviatives 19 Derviatives 20 Image derivatives - Recall: - - We don’t have an “actual” Function, must estimate - Possibility: set h = 1 - What will that look like? 21 Image derivatives - Recall: - - We don’t have an “actual” Function, must estimate - Possibility: set h = 1 - What will that look like? 22 Image derivatives - Recall: - - We don’t have an “actual” Function, must estimate - Possibility: set h = 2 - What will that look like? 23 Image derivatives - Recall: - - We don’t have an “actual” Function, must estimate - Possibility: set h = 2 - What will that look like? 24 Images are noisy! 25 But we already know how to smooth * = 26 Smooth first, then derivative 27 Smooth first, then derivative 28 Smooth first, then derivative 29 Smooth first, then derivative 30 Smooth first, then derivative 31 Smooth first, then derivative 32 Smooth first, then derivative 33 Smooth first, then derivative 34 Smooth first, then derivative 35 Smooth first, then derivative 36 Smooth first, then derivative 37 Sobel filter! Smooth & derivative 38 Image derivatives - Recall: - - Want smoothing too! 39 Finding edges - Could take derivative - Find high responses - Sobel filters! - But… 40 Finding edges - Could take derivative - Find high responses - Sobel filters! - But… - Edges go both ways - Want to find extrema 41 2nd derivative! - Crosses zero at extrema 42 Laplacian (2nd derivative)! - Crosses zero at extrema - Recall: - - Laplacian: - - Again, have to estimate f’’(x): 43 Laplacians - Laplacian: - 44 Laplacians - Laplacian: - Measures the divergence of the gradient - Flux of gradient vector field through small area 45 46 47 48 49 50 51 Laplacians - Laplacian: - 52 Laplacians - Laplacian: - 53 Laplacians - Laplacian: - 54 Laplacians - Laplacian: - 55 Laplacians - Laplacian: - - Negative Laplacian, -4 in middle - Positive Laplacian ---> 56 Laplacians also sensitive to noise - Again, use gaussian smoothing - Can just use one kernel since convs commute - Laplacian of Gaussian, LoG - Can get good approx. with 5x5 - 9x9 kernels 57 Another edge detector: - Image is a function: - Has high frequency and low frequency components - Think in terms of fourier transform - Edges are high frequency changes - Maybe we want to find edges of a specific size (i.e. specific frequency) 58 (DoG) - Gaussian is a low pass filter - Strongly reduce components with frequency f < σ - (g*I) low frequency components - I - (g*I) high frequency components - g(σ1)*I - g(σ2)*I - Components in between these frequencies - g(σ1)*I - g(σ2)*I = [g(σ1) - g(σ2)]*I 59 (DoG) - g(σ1)*I - g(σ2)*I = [g(σ1) - g(σ2)]*I - = σ = 2 σ = 1 60 (DoG) - g(σ1)*I - g(σ2)*I = [g(σ1) - g(σ2)]*I - This looks a lot like our LoG! - (not actually the same but similar) 61 DoG (1 - 0) 62 DoG (2 - 1) 63 DoG (3 - 2) 64 DoG (4 - 3) 65 Another approach: gradient magnitude - Don’t need 2nd derivatives - Just use magnitude of gradient 66 Another approach: gradient magnitude - Don’t need 2nd derivatives - Just use magnitude of gradient - Are we done? No! 67 Another approach: gradient magnitude - Don’t need 2nd derivatives - Just use magnitude of gradient - Are we done? No! 68 What we really want: line drawing 69 Canny Edge Detection - Your first image processing pipeline! - Old-school CV is all about pipelines Algorithm: - Smooth image (only want “real” edges, not noise) - Calculate gradient direction and magnitude - Non-maximum suppression perpendicular to edge - Threshold into strong, weak, no edge - Connect together components 70 Smooth image - You know how to do this, gaussians! http://bigwww.epfl.ch/demo/ip/demos/edgeDetector/ 71 Gradient magnitude and direction - Sobel filter 72 Non-maximum suppression - Want single pixel edges, not thick blurry lines - Need to check nearby pixels - See if response is highest 73 Non-maximum suppression 74 Non-maximum suppression 75 Non-maximum suppression 76 Non-maximum suppression 77 Non-maximum suppression 78 Non-maximum suppression 79 Non-maximum suppression 80 Non-maximum suppression http://bigwww.epfl.ch/demo/ip/demos/edgeDetector/ 81 Threshold edges - Still some noise - Only want strong edges - 2 thresholds, 3 cases - R > T: strong edge - R < T but R > t: weak edge - R < t: no edge - Why two thresholds? 82 Connect - Strong edges are edges! - Weak edges are edges iff they connect to strong - Look in some neighborhood (usually 8 closest) 83 Canny Edge Detection - Your first image processing pipeline! - Old-school CV is all about pipelines Algorithm: - Smooth image (only want “real” edges, not noise) - Calculate gradient direction and magnitude - Non-maximum suppression perpendicular to edge - Threshold into strong, weak, no edge - Connect together components - Tunable: Sigma, thresholds 84 Canny Edge Detection http://bigwww.epfl.ch/demo/ip/demos/edgeDetector/ 85 Canny Edge Detection http://bigwww.epfl.ch/demo/ip/demos/edgeDetector/ 86 87 So what can we do with these convolutions anyway? - Blurring - Sharpening - Edges - Features - Derivatives - Super-resolution - Classification - Detection - Image captioning -... 88 Features! - Highly descriptive local regions - Ways to describe those regions - Useful for: - Matching - Recognition - Detection 89 What makes a good feature? - Want to find patches in image that are useful or have some meaning - For objects, want a patch that is common to that object but not in general - For panorama stitching, want patches that we can find easily in another image of same place - Good features are unique! - Can find the “same” feature easily - Not mistaken for “different” features 90 How close are two patches? - Sum squared difference - Images I, J - Σx,y (I(x,y) - J(x,y))2 91 How can we find unique patches? - Say we are stitching a panorama - Want patches in image to match to other image - Need to only match one spot 92 How can we find unique patches? - Sky: bad - Very little variation - Could match any other sky 93 How can we find unique patches? - Sky: bad - Very little variation - Could match any other sky - Edge: ok - Variation in one direction - Could match other patches along same edge 94 How can we find unique patches? - Sky: bad - Very little variation - Could match any other sky - Edge: ok - Variation in one direction - Could match other patches along same edge - Corners: good! - Only one alignment matches 95 How can we find unique patches? - Want a patch that is unique in the image - Can calculate distance between patch and every other patch, lot of computation * 96 How can we find unique patches? - Want a patch that is unique in the image - Can calculate distance between patch and every other patch, lot of computation - Instead, we could think about auto-correlation: - How well does image match shifted version of itself? - ΣdΣx,y (I(x,y) - I(x+dx,y+dy))2 - Measure of self-difference (how am I not myself?) 97 Self-difference Sky: low everywhere 98 Self-difference Edge: low along edge 99 Self-difference Corner: mostly high 100 Self-difference Sky: low everywhere Edge: low along edge Corner: mostly high 101 102