Chapter 3: Image Processing PDF

Chapter 3. Image processing Contents 3.1 Pixel Processing 3.2 LSIS (Linear Shift Invariant Systems) and Convolution 3.3 Linear Image Filters 3.4 Non-Linear Image Filters  Chapter 3. Image processing 3.1 Pixel Processing We can transform the brightness value of a pixel based on...

Chapter 3. Image processing Contents 3.1 Pixel Processing 3.2 LSIS (Linear Shift Invariant Systems) and Convolution 3.3 Linear Image Filters 3.4 Non-Linear Image Filters  Chapter 3. Image processing 3.1 Pixel Processing We can transform the brightness value of a pixel based on the value itself, independently of its location or the values of other pixels in the image. It is basically a mapping of one brightness value to another brightness value or one color to another color. 𝑔 𝑥, 𝑦 = 𝑇(𝑓 𝑥, 𝑦 )  Chapter 3. Image processing 3.1 Pixel Processing The following figures illustrate an example of such transformations.  Chapter 3. Image processing 3.1 Pixel Processing  Chapter 3. Image processing 3.2 LSIS (Linear Shift Invariant Systems) and Convolution We will present this concept using one-dimensional signals before extending to multiple dimensions. Figure 1 shows an LSIS system with input f(x) and output g(x).  Chapter 3. Image processing 3.2 LSIS (Linear Shift Invariant Systems) and Convolution Properties of an LSIS Linearity: If the transformations (see below) are verified, then, the system is linear.  Chapter 3. Image processing 3.2 LSIS (Linear Shift Invariant Systems) and Convolution Properties of an LSIS Shift Invariance: Any system that satisfies linearity and shift invariance is a linear shift invariant system. The following figure illustrates that f(x) is shift invariant.  Chapter 3. Image processing 3.2 LSIS (Linear Shift Invariant Systems) and Convolution Convolution:  Chapter 3. Image processing 3.2 LSIS (Linear Shift Invariant Systems) and Convolution Convolution: If we take the product f(𝜏)h(x- 𝜏) of these two overlapping functions and integrate it from minus infinity to infinity, this gives us a single number, which is the result of the convolution at the point x.  Chapter 3. Image processing 3.2 LSIS (Linear Shift Invariant Systems) and Convolution Properties of an LSIS Convolution: To find the entire function g(x), we would flip the function h(𝜏) and then move it to minus infinity, that is the shift x in h(x- 𝜏) equals minus infinity. We then vary the shift from minus infinity to plus infinity by sliding the function h(-𝜏) over f(𝜏) from left to right. For each shift value x we find the product of the two functions and then the integral of the product. This gives us the entire function g(x), which is the result of the convolution.  Chapter 3. Image processing 3.2 LSIS (Linear Shift Invariant Systems) and Convolution Properties of an LSIS Convolution: Example of convolution.  Chapter 3. Image processing 3.2 LSIS (Linear Shift Invariant Systems) and Convolution Properties of an LSIS Convolution: Example of convolution.  Chapter 3. Image processing 3.2 LSIS (Linear Shift Invariant Systems) and Convolution Properties of an LSIS Convolution: Example of convolution.  Chapter 3. Image processing 3.2 LSIS (Linear Shift Invariant Systems) and Convolution Properties of an LSIS Convolution is LSIS : Linearity: We assume that: ,  Chapter 3. Image processing 3.2 LSIS (Linear Shift Invariant Systems) and Convolution Properties of an LSIS Convolution is LSIS : Shift Invariance We assume that: We shift the function f by the value a Where:  Chapter 3. Image processing 3.2 LSIS (Linear Shift Invariant Systems) and Convolution Properties of an LSIS Can we find f, such as g=h? This is the case of black box doing a unknown convolution filter.  Chapter 3. Image processing 3.2 LSIS (Linear Shift Invariant Systems) and Convolution Properties of an LSIS f(x) is equal to (x) such that :  Chapter 3. Image processing 3.2 LSIS (Linear Shift Invariant Systems) and Convolution Properties of an LSIS f(x) is equal to (x) such that :  Chapter 3. Image processing 3.2 LSIS (Linear Shift Invariant Systems) and Convolution Properties of convolution  Chapter 3. Image processing 3.2 LSIS (Linear Shift Invariant Systems) and Convolution Convolution 2D  Chapter 3. Image processing 3.3 Linear Image Filters In image processing, the impulse response h[i,j] is referred to as a mask, a kernel, or a filter.  Chapter 3. Image processing 3.3 Linear Image Filters the value of the output image g at pixel location [i,j] is obtained by flipping the filter h twice, overlaying it on the image f with the center of the filter at [i,j], and finding the sum of the product of the pixel values of the image and the filter in the overlap region  Chapter 3. Image processing 3.3 Linear Image Filters  Chapter 3. Image processing 3.3 Linear Image Filters  Chapter 3. Image processing 3.3 Linear Image Filters  Chapter 3. Image processing 3.3 Linear Image Filters  Chapter 3. Image processing 3.3 Linear Image Filters Gaussian Kernel: A Fuzzy Filter  Chapter 3. Image processing 3.3 Linear Image Filters Gaussian Kernel: is separable.  Chapter 3. Image processing 3.3 Linear Image Filters Convolution: How is computed (Animated image (gif)):  Chapter 3. Image processing 3.3 Linear Image Filters Convolution: Computation of the kernel and convolution using Python import math import cv2 import numpy as np import matplotlib.pyplot as plt def gkernel(l=5, sig=5): "Gaussian Kernel Creator via given length and sigma" ax = np.linspace(-(l - 1) / 2., (l - 1) / 2., l) xx, yy = np.meshgrid(ax, ax) kernel = np.exp(-0.5 * (np.square(xx) + np.square(yy)) / np.square(sig)) return kernel / np.sum(kernel)  Chapter 3. Image processing 3.3 Linear Image Filters Convolution: Computation of the kernel and convolution using Python img = cv2.imread('/content/sample_data/image.png') # Reading Image gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY) # Converting Image to grayscale g_kernel = gkernel(5,5) # Create gaussian kernel with 3x3(odd) size and sigma equals to 2 print("Gaussian Filter: ",g_kernel) # show the kernel array dst = cv2.filter2D(gray,-1,g_kernel) #convolve kernel with image img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB) # convert BGR(opencv format) to RGB format  Chapter 3. Image processing 3.3 Linear Image Filters Convolution: Computation of the kernel and convolution using Python Sigma=5, size=5, Gaussian Filter: [[0.03688345 0.03916419 0.03995536 0.03916419 0.03688345] [0.03916419 0.04158597 0.04242606 0.04158597 0.03916419] [0.03995536 0.04242606 0.04328312 0.04242606 0.03995536] [0.03916419 0.04158597 0.04242606 0.04158597 0.03916419] [0.03688345 0.03916419 0.03995536 0.03916419 0.03688345]]  Chapter 3. Image processing 3.3 Linear Image Filters Convolution: Computation of the kernel and convolution using Python plt.figure(figsize=(18, 18)) plt.subplot(131),plt.imshow(img),plt.title('Original Image') # visualize and give title plt.subplot(132),plt.imshow(gray,cmap="gray"),plt.title('GrayScaled Image') plt.subplot(133),plt.imshow(dst,cmap="gray"),plt.title('Smoothed Image with Gaussian Filter (sigma=2,3x3 Kernel)') plt.show()  Chapter 3. Image processing 3.3 Linear Image Filters Convolution: Computation of the kernel and convolution using Python Obtained result  Chapter 3. Image processing 3.3 Linear Image Filters Convolution: Some Kernels  Chapter 3. Image processing 3.4 Non-Linear Image Filters Smoothing an image to remove noise. The image in figure has some salt and pepper noise in it. If we simply apply a fuzzy filter such as a Gaussian, we can see that the noise is slightly diminished. However, what we are really doing is smearing the noise out and are not really removing it. At the same time, we are also blurring out of the edges of the coins and losing some of the details within the coins.  Chapter 3. Image processing 3.4 Non-Linear Image Filters Smoothing an image to remove noise. The image in figure has some salt and pepper noise in it. If we simply apply a fuzzy filter such as a Gaussian, we can see that the noise is slightly diminished. However, what we are really doing is smearing the noise out and are not really removing it. At the same time, we are also blurring out of the edges of the coins and losing some of the details within the coins.  Chapter 3. Image processing 3.4 Non-Linear Image Filters Smoothing an image to remove noise. The image in figure has some noise. If we simply apply a fuzzy filter such as a Gaussian, we can see that the noise is slightly diminished. However, what we are really doing is smearing the noise out and are not really removing it. At the same time, we are also blurring out of the edges of the coins and losing some of the details within the coins.  Chapter 3. Image processing 3.4 Non-Linear Image Filters Median Filter For noise removal, we take a different approach called median filtering. For each pixel, we take all the intensity values (including its own value) within a KxK window centered at the pixel and sort them.  Chapter 3. Image processing 3.4 Non-Linear Image Filters We then find the middle value of the sorted list, which is the median of all the intensity values. We simply use the median as the filtered output for the pixel. When we apply median filtering with even a small filter, say K=3, we see that the result is a significant improvement over the original image.  Chapter 3. Image processing 3.4 Non-Linear Image Filters If we use a bigger median filter (example: 11x11), the noise is reduced, but the coins are also lost. If we go to an even bigger filter, we do better in terms of the noise reduction, but even more of the details are gone.  Chapter 3. Image processing 3.4 Non-Linear Image Filters Bilateral Filter We can fix this problem by adding another term called the brightness Gaussian, which takes the difference between the brightness of the center pixel and its neighbor. If the difference is small, it will have a large value, while if it is large, it will have a low value.  Chapter 3. Image processing 3.4 Non-Linear Image Filters Bilateral Filter We can fix this problem by adding another term called the brightness Gaussian, which takes the difference between the brightness of the center pixel and its neighbor. If the difference is small, it will have a large value, while if it is large, it will have a low value.  Chapter 3. Image processing 3.4 Non-Linear Image Filters Figure. The bilateral filter smooths an input image while preserving its edges.  Chapter 3. Image processing References Sylvain Paris, Pierre Kornprobst, Jack Tumblin and Frédo Durand. Bilateral Filtering: Theory and Applications. Foundations and Trends in Computer Graphics and Vision Vol. 4, No. 1 (2008) 1–73