Computer Vision - Lecture Notes PDF

COMPUTER VISION DR. HEBA HAMDY  WHAT IS COMPUTER VISION? Computer vision is a field of artificial intelligence (AI) that enables computers and systems to derive meaningful information from digital images, videos and other visual inputs — and take actions or make recommendations...

COMPUTER VISION DR. HEBA HAMDY  WHAT IS COMPUTER VISION? Computer vision is a field of artificial intelligence (AI) that enables computers and systems to derive meaningful information from digital images, videos and other visual inputs — and take actions or make recommendations based on that information. If AI enables computers to think, computer vision enables them to see, observe and understand.  TEACHING & ASSESSMENT PLAN Week Topic 1 Introduction To Computer Vision: - What is computer vision? - Computer Vision vs. Computer Graphics - What about Image Processing? - How does computer vision work? - Computer Vision Pipeline - computer vision applications - computer vision tasks 2 Review on image processing. 2D and 3D Cartesian Coordinates - Right- and Left-handed 3D Coordinate Systems - Image Coordinate System Digital Images - Color Depth - Image Categories - Binary Images - Grayscale Images - Color Images and RGB Color Space - Assignment: Reading Assignment: Basics image processing Week Topic Filters Linear filters: - Uniform (mean) filter - Triangular filter - Gaussian filter - Non-linear filters: - Median filter A Feature Detectors - Edge Detectors - Roberts Detector - Sobel Detector - Compass Detector - Canny Detector - Line Detector Week Topic Logical & Morphological Operations Image Logic Operations - AND/OR - AND, OR, NOT (COMPLEMENT) - XOR (exclusive OR), NAND (NOT-AND) Image morphology - Erosion/ Dilation - Opening - Closing - Applications on Morphological operations Image Segmentation - Pixel based segmentation (group pixels together into regions of similarity) - Thresholding - Region splitting - Region growing (merging) - Split and merge. - Color Segmentation using K-means clustering. Reading Assignment: Segmentation - Corners detectors - Harris/Plessy Corner Detector - SIFT Detectors Midterm Exam (Theoretical & Practical)  Week Topic Feature Matching Distance Metrics - Euclidean 10 - Manhattan (City Block) - Chessboard Similarity Measures Or Correlation Techniques (Or Functions) Include: - Take Home Assignments: Appling feature Similarity Measures Machine learning - Supervised Learning - Naïve Bayes classification 11,12 - k NEAREST NEIGHBOR - Unsupervised Learning Reading Assignment: Apply Classification 13 General Revision for Final Exam 14 Final Exam (Theoretical)  GRADING POLICY Course Activity Points Quizzes (Practical) 10% Assignments 10% Project 10% Midterm Exam (Theoretical) 30% Final Exam (Theoretical ( 40% Total** 100%  COMPUTER VISION VS. COMPUTER GRAPHICS Computer vision and computer graphics: are they the same field? Computer vision is the field of science that is concerned with recognizing environments through a set of images (to help machines see). Input: images. Output: description (e.G., Locations of objects, dimensions, etc.) The ultimate goal of computer vision is to simulate the vision system in humans. COMPUTER VISION: AN EXAMPLE  COMPUTER VISION VS. COMPUTER GRAPHICS Computer graphics is the field of science that is concerned with generating visual images synthetically from descriptive information. Input: description (e.G., Locations of objects, dimensions, etc.) Output: images. Computer graphics is the opposite operation of computer vision. COMPUTER GRAPHICS: AN EXAMPLE  WHAT ABOUT IMAGE PROCESSING? Image processing is the field of science that is concerned with manipulating images. Input: images. Output: images. Most of the time, image processing techniques are used as primary steps in vision systems (e.G., To enhance the quality of images). Sometimes it is hard to draw the line between computer vision and image processing. So some topics may be categorized in both fields. IMAGE PROCESSING: AN EXAMPLE HOW DOES COMPUTER VISION WORK? Computer vision needs lots of data. It runs analyses of data over and over until it discerns distinctions and ultimately recognize images. For example, to train a computer to recognize automobile tires, it needs to be fed vast quantities of tire images and tire-related items to learn the differences and recognize a tire, especially one with no defects. Machine learning uses algorithmic models that enable a computer to teach itself about the context of visual data. If enough data is fed through the model, the computer will “look” at the data and teach itself to tell one image from another. Algorithms enable the machine to learn by itself, rather than someone programming it to recognize an image.  COMPUTER VISION PIPELINE There are typical STEPS which are found in many computer vision systems Image acquisition: a digital image is produced by one or several image sensors (e.G., Light-sensitive cameras, radar, ultra-sonic cameras, etc.)  COMPUTER VISION PIPELINE Pre-processing: it is usually necessary to process the image in order to assure that it satisfies certain assumptions before applying a computer vision method to it. For example, noise may need to be reduced or contrast may need to be enhanced.  COMPUTER VISION PIPELINE Detection/segmentation: we may need to determine which image points or regions of the image are relevant for further processing. For example, we may select a specific set of interest points or segment an image region that contains an object of interest. The position of the image segmentation phase is not fixed in the CV pipeline. It might be a part of the pre-processing phase or follow it (as in our pipeline) or be part of the feature extraction and selection phase or follow them. Segmentation is one of the oldest problems in cv and has the following aspects: ✓ partitioning an image into regions of similarity. ✓ Grouping pixels and features with similar characteristics together. ✓ Helps with selecting regions of interest within the images. These regions can contain objects of interest that we want to capture. ✓ Segmenting an image into foreground and background to apply further processing on the foreground.  COMPUTER VISION PIPELINE Feature extraction: image features are extracted. (A feature is some measurable characteristic of the input which has been found to be useful for recognition.) Examples of features are edges, corners, etc. lines and edges: these features are where sharp changes in brightness occur. They represent the boundaries of objects. Corners: these features are points of interest in the image where intersections or changes in brightness happens. These corners and edges represent points and regions of interest in the image. Brightness in this context refers to changes the in pixel intensity value.  COMPUTER VISION PIPELINE Some extracted features might be irrelevant or redundant. After feature extraction comes feature selection. Feature selection is choosing a feature subset that can reduce dimensionality with the least amount of information loss.  COMPUTER VISION PIPELINE High-level processing: from the previous step, we may use a small set of points or an image region that is assumed to be associated with a specific object. This info may be used to classify objects into categories or estimate their sizes, etc. More processing is done on the segmented images to identify more features from the image. Example: after segmentation to partition a face region, identify features on the face, such as hair style, age, and gender.  computer vision applications manufacturing: ensure that products are being positioned correctly on an assembly line. Visual auditing: look for visual compliance or deterioration in a fleet of trucks, planes, windmills, transmission or power towers , and so on. Insurance: classify claims images into different categories. Medical image processing: detect tumors. Automotive industry: object detection for safety. For example, while parking a car, a camera can detect objects and warn the driver when they get too close to them.  computer vision applications social commerce: use an image of a house to find similar homes that are for sale. social listening: track the buzz about your company on social media by looking for product logos. retail: use the photo of an item to find its price at different stores. education: use pictures to find educational material on similar subjects. public safety: automated license-plate reading.  computer vision tasks object detection and recognition: detect certain patterns within the image. Examples: ✓ detecting red eyes when taking photos in certain conditions. ✓ Face recognition. Content-based image retrieval: image retrieval from a database based on user’s image query. ✓ By using image actual feature contents such as colors, shapes, and textures ✓ not using image metadata (keywords, tags, or descriptions) optical character recognition (OCR):converting hand-written text to a digital format. COORDINATES AND IMAGES CONTENTS (c) 2018, Dr. R. Elias Images 2  2D AND 3D CARTESIAN COORDINATES 2D Cartesian coordinate system 3D Cartesian coordinate system Images 2 6 RIGHT- AND LEFT-HANDED 3D COORDINATE SYSTEMS Images Right-handed system Left-handed system 2 7 DIGITAL IMAGES  DIGITAL IMAGES Does a color have a depth?! Images 29 COLOR DEPTH Images 30 COLOR DEPTH: EXAMPLES Images 31 COLOR DEPTH: AN EXAMPLE Images 32 IMAGE CATEGORIES Images 33 BINARY IMAGES Images 34 GRAYSCALE IMAGES Images 35 COLOR IMAGES AND RGB COLOR SPACE Images 36 COLOR IMAGES AND RGB COLOR SPACE Images 37 24-BIT COLOR IMAGES Concept! Images 38 24-BIT COLOR IMAGES Concept! Images 39 8-BIT COLOR IMAGES Concept! Images 40 8-BIT COLOR IMAGES Concept! Images 41 COMPUTER VISION DR. HEBA HAMDY Lecture 2 DIGITAL FILTERS  CONTENTS ◼ In this set of slides, we will discuss what convolution and noise mean and then discuss these important smoothing filters. ◼ Linear filters: ◼ Uniform (mean) filter ◼ Triangular filter ◼ Gaussian filter ◼ Non-linear filters: ◼ Median filter ◼ Max ◼ Min  DIGITAL FILTERS ◼ An image can be filtered in the spatial or frequency domain. ◼ Filtering in the spatial domain involves convolving the image with a filter function. ◼ Convolution is used in many different operators, particularly smoothing filters and feature detectors. ◼ Filtering in the frequency domain is done by transforming the image into that domain, multiplying it with the frequency filter function and re-transforming the result into the spatial domain. WHAT IS CONVOLUTION? ◼ Convolution is a mathematical operation fundamental to many operators. ◼ The inputs to the convolution process are the image we want to perform the operation on and a 2D array of numbers called the kernel (or the Image mask). ◼ The values stored in the kernel determine the nature of the operation. kernel  HOW IS CONVOLUTION PERFORMED? ◼ The convolution is performed by sliding the kernel over the image, generally starting at the top left corner, so as to move the kernel through all the positions.  HOW IS CONVOLUTION PERFORMED? ◼ The value of an output pixel is calculated by multiplying together the kernel value and the underlying image pixel value for each of the cells in the kernel, and then adding all these numbers together. ◼ A pixel at location [x,y]T in the output image can be calculated as: where ◼ K represents a kernel of size mxn. ◼ I]y,x[ ta ytisnetni tupni eht si )y,x(1T ◼ I(x,y) is the output intensity at [x,y]T ◼ K(x,y) is the kernel applied at [x,y]T  IMAGE NOISE ◼ What is noise? ◼ Digital-camera or paper-scanned images may contain noise that degrades their quality (due to the existence of dust on the lens or faulty elements in camera, etc.). ◼ Whenever existing, noise should be reduced or dealt with as a part of a vision system.  NOISE REDUCTION BY SMOOTHING ◼ Smoothing operation is meant to reduce noise in images and/or to prepare an image for further processing; an operation that we may find useful for vision systems. ◼ There are many types of noise. Among these types of noise: ◼ Salt and pepper noise. ◼ Gaussian noise.  SALT AND PEPPER NOISE ◼ Salt and pepper noise is a random or independent noise. ◼ In this type of noise, the color (or intensity) of a noisy pixel has no relation to the color (or intensity) of the surrounding pixels. ◼ It is called salt and pepper noise because it appears as black and white dots. Salt and pepper noise  GAUSSIAN NOISE ◼ In Gaussian noise, an amount of noise is added to every part of the image where each pixel in the image will be changed from its original value by a small amount. ◼ Here the noise is dependent on the image data. ◼ The plot of the amount of distortion produces a Gaussian distribution of noise. Gaussian noise  SMOOTHING FILTERS ◼ Noise can be reduced by smoothing filters. ◼ Smoothing filters may be split into two types: ◼ Linear Filters: ◼ Uniform (mean) filter ◼ Triangular filter ◼ Gaussian filter ◼ Non-linear Filters: ◼ Median Filter. ◼ Kuwahara filter  UNIFORM (MEAN) FILTER ◼ The idea is simple; replace each pixel in the output image with the mean or average of its neighbors including itself. ◼ The convolution kernel determines the shape (rectangular or circular) and the size of the neighborhood. ◼ All the coefficients (or weights) in this kernel have identical weights. ◼ The smoothing effect depends on the kernel size. The larger the kernel the larger the smoothing effect is. ◼ We have to choose the kernel size so as to compromise between noise reduction by smoothing and blurring the image.  UNIFORM (MEAN) FILTER ◼ Notice that in all cases, the filter is normalized. ◼ Divide by the sum of kernel’s weights to normalize.  MEAN FILTER: AN APPLICATION ◼ The left image contains Gaussian noise. ◼ The right images are after smoothing using 3x3 and 5x5 mean filters. 5x5 mean filter  3X3 MEAN FILTER: AN EXAMPLE ◼ Given below is the upper left corner of an 8-bit gray-scale image. If this image is passed through a 3x3 mean filter, what would be the result of the filtering operation?  5X5 MEAN FILTER: AN EXAMPLE ◼ Given below is the upper left corner of an 8-bit gray-scale image. If this image is passed through a 5x5 mean filter, what would be the result of the filtering operation?  MEAN FILTER ◼ If I1 and I2 are two images then: ◼ Example: Show the above property on the following two upper left corners of two gray-scale images. MEAN FILTER MEAN FILTER  TRIANGULAR FILTER ◼ This filter is similar to the uniform filter in terms of calculating the average. The difference is that the weights of the kernel are of different values. ◼ Again, notice that in all cases the filter is normalized.  TRIANGULAR FILTER: AN EXAMPLE ◼ Given below is the upper left corner of an 8-bit gray-scale image. If this image is passed through a 5x5 pyramidal filter, what would be the result of the filtering operation?  TRIANGULAR FILTER: AN EXAMPLE ◼ Given below is the upper left corner of an 8-bit gray-scale image. If this image is passed through a 5x5 cone filter, what would be the result of the filtering operation?  GAUSSIAN FILTER ◼ This is also known as Gaussian blur. ◼ The kernel characterizing this filter represents the shape of a Gaussian (`bell-shaped') hump. Gaussian shape ◼ The Gaussian distribution in 1-D has the form: But we want it in where  is the standard deviation of 2D the distribution. GAUSSIAN FILTER This is a continuous Gaussian function!  GAUSSIAN FILTER ◼ The continuous Gaussian function needs to be discretized in order to be applied to digital images. Discretize  GAUSSIAN FILTER ◼ Below is a representation of a 5x5 kernel approximating a Gaussian with a s of 1.0. DISCRETE APPROXIMATION OF THE GAUSSIAN KERNELS 3X3, 5X5, 7X7  GAUSSIAN FILTER: AN APPLICATION ◼ The left image is a gray-scale image that is passed through different Gaussian filters using different  and kernel sizes.  MEDIAN FILTER ◼ This filter is used to reduce noise while preserving details. ◼ This is similar to the mean filter but instead of replacing the output pixel with the average of the neighborhood, it replaces the pixel with the median of its neighborhood. ◼ The median is calculated by sorting the values in the neighborhood and picking the middle value to replace the output pixel. ◼ If the number of pixels in the neighborhood is even, the average of The two middle numbers is used. ◼ Notice that, unlike previous filters with odd number of neighborhood pixels, the value selected will be exactly equal to one of the existing intensities.  MEDIAN FILTER: AN APPLICATION ◼ The left image contains Gaussian noise. The right image is after applying 3x3 median filter. 3x3 Median Filter  MEDIAN FILTER: AN EXAMPLE ◼ Given below is the upper left corner of an 8-bit gray-scale image. If this image is passed through a 3x3 median filter, what would be the result of the filtering operation?  MEDIAN FILTER: AN EXAMPLE ◼ Given below is the upper left corner of an 8-bit gray-scale image. If this image is passed through a 5x5 median filter, what would be the result of the filtering operation? Work it out yourself! 3X3 MEAN FILTER: AN EXAMPLE ◼ Unlike the previous filters, median filter is a non-linear filter. This means that if I1 and I2 are two imagesthen: ◼ Example: Show the above property on the following two upper left corners of two gray-scale images. MEDIAN FILTER MEDIAN FILTER  MAX FILTER SELECTS THE LARGEST VALUE WITHIN AN ORDERED WINDOW OF PIXELS VALUES AND REPLACES THE CENTRAL PIXEL WITH THE LARGEST VALUE (LIGHTEST ONE). MIN FILYER COMPUTER VISION DR. HEBA HAMDY Lecture 3  CONTENTS ❑What Is A Feature? ❑Edge Detectors ❑ Roberts Detector ❑ Sobel Detector ❑ Compass Detector ❑ Canny Detector ❑Line Detector  What Is A Feature? A feature is some measurable characteristic of the input which has been found to be useful for recognition. ✓ This is like characterizing a person by the color of his eyes, etc. ✓ Examples of useful features in images: edges, lines, junctions, corners, etc.  TYPES OF EDGES Step Edge : abrupt change from one value to a different value in the opposite side Ramp Edge: a step edge where the intensity change is not instantaneous but occurs over a finite distance.  TYPES OF EDGES Ridge Edge : - the image intensity abruptly changes value but then returns to the starting value within some short distance – generated usually by lines  TYPES OF EDGES Roof Edge : - a ridge edge where the intensity change is not instantaneous but occurs over a finite distance – generated usually by the intersection of surfaces  EDGE DETECTORS An edge can be defined as the location of a sharp transition in intensity level between two regions. Detecting edges significantly reduces the amount of data and filters out useless information, while preserving the important structural properties in an image. There are many edge detectors that vary in efficiency and complexity. in general, edge detectors tend to compute the derivative of the intensity change. This includes computing: the 1st derivative (the intensity gradient of the original data) or the 2nd derivative (the rate of change in intensity gradient)  EDGE DETECTORS ❑ An edge is a jump in intensity from one pixel to another. It has the one- dimensional shape of a ramp. ❑ Take the gradient of this signal (which, in one dimension, is just the first derivative with respect to t). ❑ Clearly, the derivative shows a maximum located at the center of the edge in the original signal. ❑ When the first derivative is at a maximum, the second derivative is zero. So, another alternative is to locate the zeroes in the second derivative.  EDGE DETECTORS 1st derivative operators (gradient detectors): Roberts detector Sobel detector Prewitt compass detector Canny detector 2nd derivative operators: zero crossing detector (laplacian of gaussian edge detector)  ROBERTS CROSS DETECTOR Roberts cross detector performs gradient measurement on an image. An output pixel is the absolute magnitude of the gradient at that pixel.  ROBERTS CROSS DETECTOR: IMPLEMENTATION How to implement Roberts cross detector? ROBERTS CROSS DETECTOR: IMPLEMENTATION  ROBERTS CROSS DETECTOR: ORIENTATION The orientation of the edge is expressed as:  ROBERTS CROSS DETECTOR Advantages: Only four pixels are involved so it is fast. Only addition and subtraction are performed so it is fast. No parameters to set. Disadvantages: the kernel is small which means that if noise exists, it will result in a false edge pixel. If a real edge is not very sharp, the operator will response weakly.  ROBERTS CROSS DETECTOR: An Application  ROBERTS CROSS DETECTOR: AN EXAMPLE Compute the Roberts cross response to the following image.  SOBEL DETECTOR Like Roberts cross operator, sobel operator performs gradient measurement on an image. An output pixel is the absolute magnitude of the gradient at that pixel. SOBEL DETECTOR: IMPLEMENTATION How to implement Sobel cross detector?  SOBEL DETECTOR: IMPLEMENTATION The absolute gradient magnitude (i.E., Edge strength) at the pixel [i,j]t can be expressed as: SOBEL DETECTOR: IMPLEMENTATION  SOBEL DETECTOR: ORIENTATION The orientation of the edge is expressed as: SOBEL DETECTOR: AN APPLICATION  SOBEL DETECTOR: AN EXAMPLE Compute the sobel response to the following image.  PREWITT COMPASS DETECTOR Prewitt compass detector is similar to roberts cross and sobel operators as it uses kernels to convolve the image with to perform edge detection. Roberts cross operator uses two kernels designed to respond maximally to edges running at 45°. Sobel operator uses two kernels designed to respond maximally to edges running vertically and horizontally. Prewitt operator uses eight kernels.  PREWITT COMPASS DETECTOR: KERNELS Each kernel is designed to respond to edges in different orientation (ranging from 0° to 315° in steps of 45°, where 0° corresponds to a vertical edge).  PREWITT COMPASS DETECTOR: MAGNITUDE AND ORIENTATION The gradient magnitude at a given pixel [i,j]t is the maximum response of the 8 kernels. the edge orientation is a value between 1 and 8 which represents the kernel that produced the maximum response.  PREWITT COMPASS DETECTOR This method is called edge template matching because we match a set of templates to the image. Compass versus roberts and sobel this operator needs to perform 8 convolutions for each pixel while roberts cross and sobel perform 2 convolutions only. On the other hand, orientation inferred by this operator is much easier comparing to the previous two operators. PREWITT COMPASS DETECTOR: AN APPLICATION An Application PREWITT COMPASS DETECTOR: AN APPLICATION  Canny Detector Canny operator performs edge detection in a number of steps. 1.Noise reduction: smooths the image. 2.Ridge detection: detects thick edges as previously done with roberts, etc. 3.Non-maximal suppression: makes the thick edges thinner. The effect of this detector is controlled by 3 parameters.  Canny Detector: 1. Noise Reduction Noise reduction: canny operator smooths the image by gaussian convolution. The size of the filter is a parameter to the algorithm. The result is a smoother image with less noise.  Canny Detector: 2. Ridge Detection Ridge detection: ◼canny applies 1st derivative operator to get a gradient magnitude image with edges as ridges. ◼Their max magnitude and corresponding orientation are detected. ◼The result is an image containing thick edges similar to the one we got with the previous operators.  Canny Detector: 3. Non-maximal Suppression Non-maximal suppression: non-maximal suppression is applied to trace along the edge direction (detected in the previous step) any tested pixel that is not on top of the ridge is set to zero. This results in thin edges How can this be done?!  Canny Detector: 3. Non-maximal Suppression Pixels not on top of the ridge are set to zero  Canny Detector: 3. Non-maximal Suppression To avoid such discontinuities, two thresholds are used t1 (high) and t2 (low). Any pixel that has a value greater than t1 is considered an edge pixel. Any pixel that has a value less than t2 is not an edge pixel. CANNY DETECTOR: AN APPLICATION Canny Detector: An Application  Laplacian Filter The laplacian filter is a measure for the 2nd derivative of an image. It highlights regions of rapid intensity change. The laplacian l(x,y) of an image with pixel intensity values i(x,y) is given by: Only one convolution kernel can approximate the 2nd derivative Three approximations to the laplacian filter are shown. You may use any of them.  Laplacian of Gaussian (LoG) Filter The laplacian filter is sensitive to noise. Thus, a gaussian filter may be applied to the image to smooth it before applying the laplacian filter. We can convolve the gaussian filter with the laplacian filter to get a hybrid filter which can be convolved with the image. This filter is called the Laplacian of Gaussian (LoG) filter. Advantages: gaussian and laplacian kernels are smaller than the image; hence, fewer arithmetic operations are performed. This results in only one kernel to convolve the image with.  Line Detectors A line feature is a thin line in an image while an edge is the boundary location between two regions. A line detector is distinguished from an edge detector and can be applied through convolution. shown are 4 kernels used to detect light lines of single pixel width against dark background at different orientations. Values should be negated to detect dark lines against light background. One may apply kernels, threshold results and add them to get final lines (binary). Note that orientation corresponds to max kernel response. COMPUTER VISION Dr. Heba Hamdy Lecture 4  AGENDA Mathematical Morphology Basic Set Theory Structuring Elements Basic morphological operations Erosion Dilation Opening Closing Basic Morphological Algorithms Boundary Extraction Region Filling INTRODUCTION Morphological operations are used as a step in image processing and analysis. It is used to modify the shape of objects in an image, by using local operations. It can be used to remove unwanted effects in segmentation post- processing Remove small objects (that is assumed to be noise) Smooth the edges of larger objects Fill holes in objects Link objects together It can be used as a part of object description and analysis Locate the boundary of objects Thin objects Locating objects with a certain structure Locating patterns in an image The operations are small, and often very fast.  MATHEMATICAL MORPHOLOGY Used to extract image components that are useful in the representation and description of region shape The applications of Morphological Image Processing are: boundaries extraction Region filling skeletons morphological filtering thinning  MATHEMATICAL MORPHOLOGY The language of Mathematical Morphology is Set Theory Sets in Mathematical Morphology represent objects in images: Binary Image the element of the set is the coordinates (x,y) of pixel belong to the object Z2 Gray-scale Image the element of the set is the coordinates (x,y) of pixel belong to the object and the gray levels Z3 BASIC SET THEORY  STRUCTURING ELEMENT (SE) Structuring Elements(SE): small sets or subimages used to probe an image under study for properties of interest. used to determine the acting range of the operations. The matrix dimensions specify the size of the structuring element. The pattern of ones and zeros specifies the shape of the structuring element. for each SE, define origin(black dot). When SE is symmetric and no dot is show the origin is at the center of symmetry.  EXAMPLES: STRUCTURING ELEMENTS (1) origin  BASIC IDEA in parallel for each pixel in binary image: check if SE is ”satisfied” output pixel is set to 0 or 1 depending on used operation The structuring element is said to fit the image if, for each of its pixels set to 1, the corresponding image pixel is also 1. Similarly, a structuring element is said to hit, an image if, at least for one of its pixels set to 1 the corresponding image pixel is also 1. Zero-valued pixels of the structuring element are ignored  BASIC MORPHOLOGICAL OPERATIONS Erosion removal of structures of certain shape and size, given by SE Dilation filling of holes of certain shape and size, given by SE  EROSION is used for shrinking of element A by using element B Does the structuring element fit the set? Erosion for Sets A and B in Z2 , is defined by the following equation: This equation indicates that the erosion of A by B is the set of all points z such that B, translated by z, is contained in A.  EROSION Erosion can be used to: Shrinks or thins objects in binary images Remove image components Erosion is a morphological filtering operation in which image details smaller than the structuring elements are filtered(removed)  EROSION EXAMPLE Original Image Processed Image With Eroded Pixels Structuring Element  EROSION EXAMPLE Original Image Processed Image Structuring Element EROSION EXAMPLE EROSION: EXAMPLE  EROSION EXAMPLE Original image Erosion by 3*3 Erosion by 5*5 square structuring square structuring element element WATCH OUT: IN THESE EXAMPLES A 1 REFERS TO A BLACK PIXEL!  EROSION EXAMPLE After erosion Original with a disc of image radius 10 After erosion After erosion with a disc of with a disc of radius 5 radius 20  DILATION IS USED FOR EXPANDING AN ELEMENT A BY USING STRUCTURING ELEMENT B DILATION OF A BY B AND IS DEFINED BY THE FOLLOWING EQUATION: DOES THE STRUCTURING ELEMENT HIT THE SET? THE DILATION OF A BY B IS THE SET OF ALL DISPLACEMENTS Z, SUCH THAT AND A OVERLAP BY AT LEAST ONE ELEMENT.  DILATION Dilation can be used to: Grows or thickens object in a binary image Bridging gaps Fill small holes of sufficiently small size  DILATION EXAMPLE Original Image Processed Image Structuring Element  DILATION EXAMPLE Original Image Processed Image With Dilated Pixels Structuring Element DILATION EXAMPLE DILATION: EXAMPLE DILATION : BRIDGING GAPS 22 EXAMPLES : BLOB DETECTION/ SEPARATION(EROSION)  DILATION EXAMPLE 2 Original image Dilation by 3*3 Dilation by 5*5 square structuring square structuring element element Watch out: In these examples a 1 refers to a black pixel! Examples of Dilation (3) EXAMPLES OF DILATION (4)  EDGE DETECTION Edge detection 1.Dilate input image 2.Subtract the dilated image from input image. 3.Edges remain!  COMBINING EROSION AND DILATION WANTED: remove structures / fill holes without affecting remaining parts SOLUTION: combine erosion and dilation (using same SE)  OPENING Erosion followed by Dilation with the same SE, Eliminates protrusions Smooths contour OPENING OPENING  OPENING : ELIMINATING IRRELEVANT DETAIL structuring element B = 13x13 pixels of gray level 1  CLOSING Dilation followed by Erosion smooth contour fuse narrow breaks and long thin gulfs eliminate small holes fill gaps in the contour CLOSING CLOSING EXAMPLE: SMALL HOLE REMOVAL USING CLOSING OPENING & CLOSING Application: Morphological Filtering APPLICATION: CORNER DETECTION  BASIC MORPHOLOGICAL ALGORITHMS Boundary Extraction Region Filling Extraction of connected components (Assignment)  BOUNDARY EXTRACTION First erode A by B, then make set difference between A and the erosion The thickness of the contour depends on the size of constructing object –B EXAMPLE  REGION FILLING This algorithm is based on a set of dilations, complementation and intersections p is the point inside the boundary, with the value of 1 X(k) = (X(k-1) dilate B) conjunction with complemented A The process stops when X(k) = X(k-1) The result that given by union of A and X(k), is a set contains the filled set and the boundary EXAMPLE  SOME BASIC MORPHOLOGICAL ALGORITHMS Extraction of connected components Central to many automated image analysis applications. Let a be a set containing one or more connected components, and form an array x0 (of the same size as the array containing a) whose elements are 0s, except at each location known to correspond to a point in each connected component in a, which is set to 1. 10/2/2024 51  SOME BASIC MORPHOLOGICAL ALGORITHMS Extraction of connected components Central to many automated image analysis applications. X k = ( X k −1 + B )  A B : structuring element until X k = X k -1 10/2/2024 52 10/2/2024 53 10/2/2024 54 EXAMPLE COMPUTER VISION DR. HEBA HAMDY Lecture 5 AGENDA ⚫ What is Segmentation Segmentation techniques: 1. Thresholding 2. Edge-based Segmentation 3. Region-based Segmentation Region Merging Region Splitting Merge and Split Computer Vision Segmentation Segmentation ⚫ Goal: to divide an image into parts that have a strong correlation with objects or areas of the real world, mainly objects and background. All pixels in a region share a common property, like intensity, texture. A complete segmentation of an image R is a set of regions R1, R2, …, Rs Computer Vision Segmentation  SEGMENTATION ⚫ Segmentation may be ▪ Complete segmentation - set of disjoint regions uniquely corresponding with objects in the input image ▪ Partial segmentation - regions do not correspond directly to image objects Computer Vision Segmentation THRESHOLDING SEGMENTATION TECHNIQUE ⚫ Basic, Band, Multi, Semi-Thresholding ⚫ Threshold detection methods ⚫ Multi-spectral Thresholding ⚫ Algorithms Iterative (optimal) Threshold Selection  THRESHOLDING: SIMPLE FORM Thresholding or global thresholding is used to segment an image according to different intensities (e.g., In case you want to extract objects out of a background). Thresholding: each pixel is compared with the threshold: if its intensity is greater than the threshold, it is set to black. If its intensity is less than the threshold, it is set to white.  BASIC THRESHOLDING ⚫ Basic thresholding of an input image f to an output image g is as follows: g(i,j) = 1 for f(i,j) >T g(i,j) = 0 for f(i,j) T=4 —> 1 If p ≤ T=4 —>0 GLOBAL THRESHOLD G1 = {7,5,5,6,6,7,5,5,6,7,5,5,5, 6,7,5,7} G2={3,1,2,3,2,0,4,2,0,4,2,2,3,3,4, 3,3,0,4} ml = 99 / 17 = 5.82 m2 = 45/19 = 2.36 Tnew = 5.82+2.36/2 = 8.18/2 = 4.09 ~ 4  GLOBAL THRESHOLD Segment the following sub image use the Intensity based segmentation (global threshold)?  ADVANTAGES ⚫ Simplest segmentation process since Many objects or image regions are characterized by constant reflectivity or light absorption of their surface. ⚫ computationally inexpensive and fast and can easily be done in real time ⚫ When to use Thresholding: − Objects don’t touch each other − Objects ’gray levels are clearly distinct from background ’gray level. Computer Vision Segmentation  BAND THRESHOLDING ⚫ There are several modifications on basic thresholding. They include: 1. Band thresholding: g(i,j) = 1 for f(i,j) f(i,j) ∈ D g(i,j) = 0 otherwise ⚫ used e.g. in microscopic blood cells Computer Vision Segmentation  MULTI- THRESHOLDING 2. Multi-thresholding, using limited set of array levels: g(i,j) = 1 for f(i,j) ∈ D1 g(i,j) = 2 for f(i,j) ∈ D2 g(i,j) = 3 for f(i,j) ∈ D3... ∈ Dn g(i,j) = n for f(i,j) g(i,j) = 0 otherwise Computer Vision Segmentation  SEMI-THRESHOLDING 3. Semi-thresholding, masks out background g(i,j) = f(i,j) for f(i,j) >= T