gabor_wavelet (1).pdf

Gabor Filters and Wavelet Analysis Amirhassan Monajemi, 2024  We are going to review MSMD methods in computer vision. Introduction We will focus on Wavelet analysis and Gabor...

Gabor Filters and Wavelet Analysis Amirhassan Monajemi, 2024  We are going to review MSMD methods in computer vision. Introduction We will focus on Wavelet analysis and Gabor filtering. In this course, Wavelet will be examined, but Gabor will not. Gabor & Wavelet 2  Multi-Scale / Multi-Directional Image Analytics It involves analyzing images at different scales (resolutions) and across various orientations (directions) to extract more detailed and comprehensive information. This technique is especially useful for capturing features that may vary in size and orientation within the image, such as edges, textures, or patterns. Gabor & Wavelet 3 Multi-Scale / Multi-Directional Image Analytics Scale refers to the size of the features in the image. Multi- scale analysis involves analyzing the image at various levels of resolution. This allows the detection of both fine details and broader/ larger/ coarser structures. In practice, this is achieved by applying filters (such as Wavelets) that reduce the image’s resolution in a controlled manner. Features like edges, textures, or patterns that may be hard to detect at one scale, may become more apparent at another. Also, Scale refers to the frequency of Wavelet (AKA kernels or filters) that we apply to the image to analyze it. Gabor & Wavelet 4  Multi-Scale / Multi-Directional Image Analytics Directionality refers to the orientation of features in an image, such as edges or lines. Multi-directional analysis involves analyzing the image from various angles or directions. This is useful for detecting oriented patterns such as horizontal, vertical, or diagonal edges. By applying filters that are sensitive to specific orientations (e.g., Gabor filters or directional wavelets), you can capture features with varying orientations within the image. 5 Gabor & Wavelet  A Trade-Off… Analyzing the image in the spatial domain: Good spatial information and resolution, but no information about the frequency domain components. Analyzing the image in the transform domain (e.g. Hadamard): Good information about the frequency domain components, but no clue about the spatial domain. Analyzing the image using MSMD approaches: A trade- off where you can keep both the spatial domain and the frequency domain information. Gabor & Wavelet 6  A stationary image is one where the statistical properties (such as the mean, variance, and texture characteristics) remain constant throughout the image. This means that the image has a homogeneous Stationary vs. structure or pattern, and the same statistical characteristics (like intensity distribution, frequency Non- content, or texture) can be observed across all parts of the image. Stationary Otherwise, the image will be non-stationary. Practically, real-world images are rarely truly stationary, but synthetic or idealized images used in research (such as random noise images) can be stationary. Gabor & Wavelet 7  A wide-sense stationary (WSS) image is one where certain statistical properties of the image remain consistent, but not all. Specifically, an image is considered wide-sense stationary if its mean is constant, and its autocorrelation (or covariance) function depends only on the relative spatial positions (differences between coordinates), not on the absolute positions in the image. Wide-Sense An image with a repeating texture or pattern that has Stationary no large-scale intensity changes could be assumed as wide-sense stationary, where the mean intensity stays constant, and the correlation between pixels is uniform across the image. A synthetic image with random noise, such as Gaussian noise, often exhibits wide-sense stationary properties because the noise has a constant mean, and the autocorrelation depends only on the distance between pixels, not their absolute positions 8 Gabor & Wavelet  Scaling-- value of “stretch” Scaling a wavelet simply means stretching (or compressing) it. f(t) = sin(t) f(t) = sin(2t) f(t) = sin(3t) scale factor 1 scale factor 2 scale factor 3 Attention: In some functions in different programming languages, the scale parameter is the wavelength= 1/frequency. E.g., getGaborKernel function in Python OpenCV. In that function, the scale parameter is the wavelength of the Wavelet, not Gabor & Wavelet its frequency. So, to have a high frequency Wavelet, we should keep that parameter small and vice versa. 9  More on Scaling It lets you either narrow down the frequency band of interest, or determine the frequency content in a narrower time interval Scaling = frequency band Good for non-stationary data Low scalea Compressed wavelet Rapidly changing details High frequency. High scale a Stretched wavelet  Slowly changing, coarse features  Low frequency Gabor & Wavelet 10  Scale is (sort of) like frequency Small scale -Rapidly changing details, -Like high frequency Large scale -Slowly changing details Gabor & Wavelet -Like low frequency 11  Scale is (sort of) like frequency The scale factor works exactly the same with wavelets. The smaller the scale factor, the more "compressed" the wavelet. Gabor & Wavelet 12  Shifting Shifting a wavelet simply means delaying (or hastening) its onset. Mathematically, delaying a function f(t) by k is represented by f(t-k) Gabor & Wavelet 13  Shifting C = 0.0004 14 C = 0.0034 Gabor & Wavelet  Directional Filtering The image shows a bookshelf, with more vertical edges, so higher power on vertical edge detector filter response. Gabor & Wavelet 15  Ordinary Gradient Filters are Wavelets In fact, filters that we have used so far can be considered as wavelets with different frequencies and directions. filters here are all spatial domain filters. Gabor & Wavelet 16 Larger Wavelet Matrices A 17x17 Gabor filter, direction = 60 We can go for wavelets with different frequencies or keep the wavelet frequency constant but change the frequency contents of the image under analysis (IUA). To do that, we apply LPF on IUA. Gabor & Wavelet 17 Wavelet Analysis Gabor & Wavelet 18  Multi-scale Decomposition: In traditional wavelet analysis, the image is decomposed into multiple scales using wavelets, which act as band-pass filters at different frequencies. This captures high-frequency details at finer scales and low-frequency components at coarser scales. Low-pass Filtering: If you apply a low-pass filter step- How to do Wavelet by-step (such as Gaussian filtering) to reduce the resolution of the image, this is similar to generating a pyramid of low-pass filtered images (coarse representations). Bullets show the Wavelet analysis of an image. Each step reduces the spatial resolution, and then you can apply a fixed wavelet at each level. Fixed Wavelet: Applying a few fixed Wavelet filters (usually Gradient filters) at different levels of low-pass filtered images can give you a multi-resolution representation. Different Wavelets for different scales to better capture fine and coarse details. Gabor & Wavelet Results of highpass filtering applied on the image 19 should be integrated into the Wavelet analysis, too.  Code Analysis Wavelet3.ipynb 1- definition of 4 gradient filters, which will be our Wavelets here. 2- reading the image, showing its size and power Gabor & Wavelet 20  Code Analysis Wavelet3.ipynb 3- Building the Gaussian/Low frequency pyramid in imgl list 4- Building the Laplacian/High frequency pyramid in imgh list. 5- Filtering LF levels with 4 gradient filters and save the results in 4 respective lists, fl1, fl2, fl3, and fl4. Gabor & Wavelet 21  Code Analysis Wavelet3.ipynb 6- We do that for 8 levels. Here is the power of different channels of this Wavelet analysis. Gabor & Wavelet 22  Code Analysis Wavelet3.ipynb 8 levels of the Gaussian/ Low frequency pyramid. Sizes are scaled. In reality the size of the last level is just Gabor & Wavelet 23  Code Analysis Wavelet3.ipynb 8 levels of the Laplacian/ High frequency pyramid. Sizes are scaled. In reality the size of the last level is just Gabor & Wavelet 24 Gabor Filtering Gabor & Wavelet 25  Gabor Filters If we go for larger filter size, e.g. 17x17 like what you can see here, the frequency of the wavelet can get higher. Also, filters can have more directions. A 17x17 Gabor filter, direction = 90 Gabor & Wavelet 26  Gabor filters are indeed sine or cosine wavelets modulated by a Gaussian envelope. These filters are widely used in image processing and computer vision tasks, particularly for edge detection, texture analysis, and feature extraction. Sine or Cosine Wavelet: The core of the Gabor filter is a sinusoidal plane wave (either sine or cosine), which is Gabor Filters responsible for capturing the frequency information in an image. This wave has a specific frequency and The combination of the sinusoidal wave and the Gaussian envelope allows orientation that allows it to respond to particular Gabor filters to be effective in detecting textures or patterns in the image. localized frequency content in an image, such as edges, textures, and Gaussian Envelope: The sinusoidal wave is modulated orientations. by a Gaussian function, which limits the spatial extent of the filter. This ensures that the filter is localized in both the spatial domain (image space) and frequency domain. The Gaussian envelope makes the Gabor filter more sensitive to specific regions of the image, while Gabor & Wavelet the sinusoidal component captures the frequency 27 information.  Gabor Filters Equation 2d Gabor filter in spatial domain. Gabor & Wavelet 28 Gabor Filters Cosine Wavelet, Gaussian envelope, and modulation, 1d Gabor filters. Gabor & Wavelet 29 Gabor Filters Cosine Wavelet, Gaussian envelope, and modulation, 1d Gabor filters. Gabor & Wavelet 30 Gabor Filters Cosine Wavelet, Gaussian envelope, and modulation, 2d Gabor filters. Gabor & Wavelet 31 Gabor Filters Cosine Wavelet, Gaussian envelope, and modulation, 2d Gabor filters. Gabor & Wavelet 32  Gabor Filters in Python Gabor1.ipynb getGaborKernel function to build a Gabor filter filter2D to convolve the filter with image Gabor & Wavelet 33  Test Image 20230515_110825.jpg Gabor & Wavelet 34  Gabor Filters A 17x17 Gabor filter, direction = 0 In the getGaborKernal function, these 2 parameters control the frequency (actually, normalized period) and direction of the wavelet. Gabor & Wavelet 35  Gabor Filters A 17x17 Gabor filter, direction = 90, Wavelet period length= 9 cv2.getGaborKernel((17, 17), 8.0, np.pi/2, 9, 0.5, 0, ktype=cv2.CV_32F) Gabor & Wavelet 36  Gabor Filters A 17x17 Gabor filter, direction = 90, Wavelet period length= 18 cv2.getGaborKernel((17, 17), 8.0, np.pi/2, 18, 0.5, 0, ktype=cv2.CV_32F) Gabor & Wavelet 37  Gabor Filters A 17x17 Gabor filter, direction = 0, Wavelet period length= 18 cv2.getGaborKernel((17, 17), 8.0, 0, 18, 0.5, 0, ktype=cv2.CV_32F) Gabor & Wavelet 38  Gabor Filters This parameter controls the width/ Standard Deviation of the Gaussian envelope. Compare the filter and results with Slide 31. Other parameters are equal, but envelope width decreased from 8 to 2. Gabor & Wavelet 39  Gabor Filters Gaussian width parameter, again cv2.getGaborKernel((17, 17), 2.0, np.pi/2, 2, 0.5, 0, ktype=cv2.CV_32F) Compare it with Slide 35. Gabor & Wavelet 40  Gabor Filters Phase parameter, highlighted, controls the phase of the wavelet. cv2.getGaborKernel((17, 17), 8.0, np.pi/2, 18, 0.5, np.pi/2,ktype=cv2.CV_ 32F) Compare it with Slide 37. Gabor & Wavelet 41  References http://www.wavelet.org http://www.multires.caltech.edu/teaching Gabor & Wavelet 42