COMP9517_24T2W1_Image_Processing_Basics.pdf
Document Details
![FastGrowingJackalope](https://quizgecko.com/images/avatars/avatar-12.webp)
Uploaded by FastGrowingJackalope
UNSW Sydney
2024
Tags
Related
- COMP9517 Computer Vision 2024 Term 2 Week 2 Image Processing Part 1 PDF
- COMP9517 Computer Vision Image Segmentation Part 2 PDF
- Introduction to Computer Vision - Fall 2024 - Toronto Metropolitan University PDF
- Introduction to Computer Vision PDF Fall 2024
- Digital Image Processing 4th Edition - Global Edition PDF
- Week 2 Module 1: Introduction to Digital Image Processing PDF
Full Transcript
COMP9517 Computer Vision 2024 Term 2 Week 1 Professor Erik Meijering Image Processing Basics What is image processing? Image processing = image in > image out Aims to suppress distortions and enhance relevant information Used to p...
COMP9517 Computer Vision 2024 Term 2 Week 1 Professor Erik Meijering Image Processing Basics What is image processing? Image processing = image in > image out Aims to suppress distortions and enhance relevant information Used to prepare images for further analysis and interpretation Image analysis = image in > features out Computer vision = image in > interpretation out Copyright (C) UNSW COMP9517 24T2W1 Image Processing Basics 2 Types of image processing Two main types of image processing operations: β Spatial domain operations (in image space) Next week β Transform domain operations (mainly in Fourier space) Two main types of spatial domain operations: Today β Point operations (intensity transformations on individual pixels) β Neighbourhood operations (spatial filtering on groups of pixels) Copyright (C) UNSW COMP9517 24T2W1 Image Processing Basics 3 Topics and learning goals Describe the workings of basic point operations Contrast stretching, thresholding, inversion, log/power transformations Understand and use the intensity histogram Histogram specification, equalization, matching Define arithmetic and logical operations Summation, subtraction, AND/OR, averaging Copyright (C) UNSW COMP9517 24T2W1 Image Processing Basics 4 Spatial domain operations General form of spatial domain operations ππ π₯π₯, π¦π¦ = ππ ππ π₯π₯, π¦π¦ where ππ π₯π₯, π¦π¦ is the input image ππ π₯π₯, π¦π¦ is the processed image ππ is the operator applied at (π₯π₯, π¦π¦) Copyright (C) UNSW COMP9517 24T2W1 Image Processing Basics 5 Spatial domain operations Point operations: ππ operates on individual pixels ππ: β βΆ β ππ π₯π₯, π¦π¦ = ππ ππ π₯π₯, π¦π¦ Neighbourhood operations: ππ operates on multiple pixels ππ: βππ βΆ β ππ π₯π₯, π¦π¦ = ππ ππ π₯π₯, π¦π¦ , ππ π₯π₯ + 1, π¦π¦ , ππ π₯π₯ β 1, π¦π¦ , β¦ Copyright (C) UNSW COMP9517 24T2W1 Image Processing Basics 6 Point operations Input Output image image Copyright (C) UNSW COMP9517 24T2W1 Image Processing Basics 7 Contrast stretching Input Output intensity ππ Output πΏπΏ π»π» Input intensity Copyright (C) UNSW COMP9517 24T2W1 Image Processing Basics 8 Contrast stretching Produces images of higher contrast Puts values below πΏπΏ in the input to the minimum (black) in the output Puts values above π»π» in the input to the maximum (white) in the output Linearly scales values between πΏπΏ and π»π» (inclusive) in the input to between the minimum (black) and the maximum (white) in the output Copyright (C) UNSW COMP9517 24T2W1 Image Processing Basics 9 Intensity thresholding Input Output intensity Output ππ Threshold level Input intensity Copyright (C) UNSW COMP9517 24T2W1 Image Processing Basics 10 Intensity thresholding Limiting case of contrast stretching Produces binary images of gray-scale images Puts values below the threshold to black in the output Puts values equal/above the threshold to white in the output Popular method for image segmentation (discussed later) Useful only if object and background intensities are very different Result depends strongly on the threshold level (user parameter) Copyright (C) UNSW COMP9517 24T2W1 Image Processing Basics 11 Intensity thresholding Input Output intensity Output ππ Threshold level Input intensity Copyright (C) UNSW COMP9517 24T2W1 Image Processing Basics 12 Intensity thresholding Input Output intensity Output ππ Threshold level Input intensity Copyright (C) UNSW COMP9517 24T2W1 Image Processing Basics 13 Intensity thresholding Input Output intensity Output ππ Threshold level Input intensity Copyright (C) UNSW COMP9517 24T2W1 Image Processing Basics 14 Automatic intensity thresholding Otsuβs method for computing the threshold automatically https://doi.org/10.1109/TSMC.1979.4310076 Exhaustively searches for the threshold minimising the intra-class variance 2 ππππ = ππ0 ππ02 + ππ1 ππ12 Equivalent to maximising the inter-class variance (much faster to compute) πππ΅π΅2 = ππ0 ππ1 ππ0 β ππ1 2 Here, ππ0 is the fraction of pixels below the threshold (class 0), ππ1 is the fraction of pixels equal to or above the threshold (class 1), ππ0 and ππ1 are the mean intensities of pixels in class 0 and class 1, ππ02 and ππ12 are the intensity variances, and ππ0 + ππ1 = 1 and ππ02 + ππ12 = ππ 2 Copyright (C) UNSW COMP9517 24T2W1 Image Processing Basics 15 Otsu thresholding example Copyright (C) UNSW COMP9517 24T2W1 Image Processing Basics 16 Automatic intensity thresholding IsoData method for computing the threshold automatically 1. Select an arbitrary initial threshold π‘π‘ 2. Compute ππ0 and ππ1 with respect to the threshold 3. Update the threshold to the mean of the means: π‘π‘ = ππ0 + ππ1 /2 4. If the threshold changed in Step 3, go to Step 2 Upon convergence, the threshold is midway between the two class means Copyright (C) UNSW COMP9517 24T2W1 Image Processing Basics 17 Isodata thresholding example Copyright (C) UNSW COMP9517 24T2W1 Image Processing Basics 18 Multilevel thresholding Input Output intensity ππ Output Threshold #1 Threshold #2 Input intensity Copyright (C) UNSW COMP9517 24T2W1 Image Processing Basics 19 Intensity inversion Input Output intensity ππ Output Input intensity Copyright (C) UNSW COMP9517 24T2W1 Image Processing Basics 20 Intensity inversion examples βAssessment of grayscale inverted images in addition to standard images facilitates the detection of microcalcification.β https://doi.org/10.1186/s12880-017-0196-6 Copyright (C) UNSW COMP9517 24T2W1 Image Processing Basics 21 Log transformation Definition of log transformation π π = ππ log 1 + ππ where ππ is the input intensity, π π is the output Output gray level π π intensity, and ππ is a constant β Maps a narrow input range of low gray-level values into a wider range of output values, and vice versa for higher gray-level values β Also compresses the dynamic range of images with large variations in pixel values (such as Fourier spectra, to be discussed later) Input gray level ππ Copyright (C) UNSW COMP9517 24T2W1 Image Processing Basics 22 Power transformation Definition of power transformation π π = ππ ππ πΎπΎ where ππ and πΎπΎ are constants Output gray level π π β Similar to (inverse) log transformation β Represents a family of transformations by varying πΎπΎ β Many devices respond according to a power law β Example power transformation: gamma correction β Useful for general-purpose contrast manipulation Input gray level ππ Copyright (C) UNSW COMP9517 24T2W1 Image Processing Basics 23 Power transformation examples ππ = 1 Input πΎπΎ = 3 πΎπΎ = 4 πΎπΎ = 5 Copyright (C) UNSW COMP9517 24T2W1 Image Processing Basics 24 Piecewise linear transformations Complementary to other transformation methods Enable more fine-tuned design of transformations Output gray level π π Can have very complex shapes Requires more user input Input gray level ππ Copyright (C) UNSW COMP9517 24T2W1 Image Processing Basics 25 Piecewise contrast stretching One of the simplest piecewise linear transformations Increases the dynamic range of gray levels in images Used in display devices or recording media to span full range Input Transformed Thresholded Output gray level π π Input gray level ππ Copyright (C) UNSW COMP9517 24T2W1 Image Processing Basics 26 Gray-level slicing Transform 1 Transform 2 Used to highlight a specific range of gray levels Two different slicing approaches: 1) High value for all gray levels in a range of interest and low value for all others (produces a binary image) 2) Brighten a desired range of gray levels while preserving background and other gray-scale tones of the image Input Result of 1 Copyright (C) UNSW COMP9517 24T2W1 Image Processing Basics 27 Bit-plane slicing Input Highlights contribution to total image by specific bits An image with n bits/pixel has n bit-planes Can be useful for image compression Image Bit-planes Bit-plane 7 (most significant) 151 = 1 0 0 1 0 Bit-plane 0 (least significant) 1 1 1 Copyright (C) UNSW COMP9517 24T2W1 Image Processing Basics 28 Histogram of pixel values For every possible gray-level value, count the number of pixels having that value, and plot the pixel counts as a function of gray level πΏπΏ = 28 = 256 Count β(ππ) ππ = #pixels πΏπΏβ1 β(ππ) = ππ ππ=0 Normalized histogram = probability function 1 β(ππ) = ππ(ππ) ππ 8-bit image Level Copyright (C) UNSW COMP9517 24T2W1 Image Processing Basics 29 Histogram based thresholding Triangle method for computing the threshold automatically 1. Find the histogram peak (ππππ , βππ ) and βππ the highest gray level point (ππππ , βππ ) ππ(ππ) 2. Construct a straight line ππ(ππ) from the β(ππ) ππ(ππ) peak to the highest gray level point 3. Find the gray level ππ for which the βππ distance ππ ππ β β(ππ) is the largest 0 ππππ ππ ππππ 255 Copyright (C) UNSW COMP9517 24T2W1 Image Processing Basics 30 Comparison of thresholding methods Image Histogram Otsu IsoData Triangle Copyright (C) UNSW COMP9517 24T2W1 Image Processing Basics 31 Histogram processing Histogram equalization Aim: To get an image with equally distributed intensity levels over the full intensity range Histogram specification (also called histogram matching) Aim: To get an image with a specified intensity distribution, determined by the shape of the histogram Copyright (C) UNSW COMP9517 24T2W1 Image Processing Basics 32 Histogram equalization Enhances contrast for intensity values near histogram maxima and decreases contrast near histogram minima Histogram bins are much more βequalβ here Copyright (C) UNSW COMP9517 24T2W1 Image Processing Basics 33 Histogram equalization Let ππ β 0, πΏπΏ β 1 represent pixel values (intensities, gray levels) ππ = 0 represents black and ππ = πΏπΏ β 1 represents white Consider transformations π π = ππ ππ , 0 β€ ππ β€ πΏπΏ β 1, satisfying 1) ππ(ππ) is single-valued and monotonically increasing in 0 β€ ππ β€ πΏπΏ β 1 This guarantees that the inverse transformation ππ β1 (π π ) exists 2) 0 β€ ππ ππ β€ πΏπΏ β 1 for 0 β€ ππ β€ πΏπΏ β 1 This guarantees that the input and output ranges will be the same Copyright (C) UNSW COMP9517 24T2W1 Image Processing Basics 34 Histogram equalization (continuous case) Consider ππ and π π as continuous random variables over 0, πΏπΏ β 1 with PDFs ππππ (ππ) and πππ π (π π ) If ππππ (ππ) and ππ(ππ) are known and ππ β1 (π π ) satisfies monotonicity, then, from probability theory ππππ πππ π π π = ππππ (ππ) https://www.cl.cam.ac.uk/teaching/2003/Probability/prob11.pdf ππππ ππ Let us choose: π π = ππ ππ = (πΏπΏ β 1) β«0 ππππ ΞΎ ππΞΎ This is the CDF (cumulative distribution function) of ππ which satisfies conditions (1) and (2) ππππ ππππ(ππ) ππ ππ Now: = = πΏπΏ β 1 β«0 ππππ ΞΎ ππΞΎ = (πΏπΏ β 1)ππππ (ππ) ππππ ππππ ππππ 1 1 Therefore: πππ π (π π ) = ππππ(ππ) = for 0 β€ π π β€ πΏπΏ β 1 (uniform distribution) πΏπΏβ1 ππππ ππ πΏπΏβ1 Copyright (C) UNSW COMP9517 24T2W1 Image Processing Basics 35 Histogram equalization (discrete case) For discrete values we get probabilities and summations instead of PDFs and integrals: ππππ(ππππ) = ππππ /ππππ for ππ = 0, 1, β¦ , πΏπΏ β 1 where ππππ is the total number of pixels in image, ππππ is the number of pixels with gray level ππππ , and πΏπΏ is the total number of gray levels in the image πΏπΏβ1 ππ Thus: π π ππ = ππ ππππ = (πΏπΏ β 1) βππππ=0 ππππ ππππ = β ππ for ππ = 0, 1, β¦ , πΏπΏ β 1 ππππ ππ=0 ππ This transformation is called histogram equalization However, for discrete images, applying a single mapping function does not give a truly uniform distribution, and adaptive approaches (multiple mapping functions) are needed Copyright (C) UNSW COMP9517 24T2W1 Image Processing Basics 36 Constrained histogram equalization Input Full histogram equalization (slope of ππ(ππ) is unconstrained) ππ(ππ) β(ππ) β(ππ) ππ(ππ) β(ππ) Constrained histogram equalization (slope of ππ ππ is constrained) Copyright (C) UNSW COMP9517 24T2W1 Image Processing Basics 37 Histogram matching (continuous case) Assume that ππ and π π are continuous and πππ§π§ (π§π§) is the target distribution for the output image From our previous analysis we know the following transformation results in a uniform distribution: ππ ππππ (ππ) π π = ππ ππ = (πΏπΏ β 1) β«0 ππππ ΞΎ ππΞΎ π π = ππ(ππ) Now we can define a function πΊπΊ(π§π§) as: πππ π (π π ) β1 ππ = ππ (π π ) π§π§ πΊπΊ π§π§ = πΏπΏ β 1 β«0 πππ§π§ ΞΎ ππΞΎ = π π π§π§ = πΊπΊ β1 (π π ) Therefore: πππ§π§ (π§π§) π§π§ = πΊπΊ β1 π π = πΊπΊ β1 ππ(ππ) π π = πΊπΊ(π§π§) Copyright (C) UNSW COMP9517 24T2W1 Image Processing Basics 38 Histogram matching (discrete case) For discrete image values we can write: πΏπΏβ1 ππ π π ππ = ππ ππππ = (πΏπΏ β 1) βππππ=0 ππππ ππππ = β ππ ππππ ππ=0 ππ ππ = 0, 1, β¦ , πΏπΏ β 1 ππ And: πΊπΊ π§π§ππ = (πΏπΏ β 1) βππ=0 πππ§π§ (π§π§ππ ) Therefore: π§π§ππ = πΊπΊ β1 (π π ππ ) Copyright (C) UNSW COMP9517 24T2W1 Image Processing Basics 39 Histogram matching example https://automaticaddison.com/tag/image-processing/page/3/ Input Target Output Matching done for each colour channel (R,G,B) Copyright (C) UNSW COMP9517 24T2W1 Image Processing Basics 40 Arithmetic and logical operations Defined on a pixel-by-pixel basis between two images Input 1 Input 2 Output + = β β / ^ AND OR XOR β¦ Copyright (C) UNSW COMP9517 24T2W1 Image Processing Basics 41 Arithmetic and logical operations Useful arithmetic operations include addition and subtraction Input 1 Output - Input 2 Copyright (C) UNSW COMP9517 24T2W1 Image Processing Basics 42 Arithmetic and logical operations Useful logical operations include bitwise AND and OR Input Mask Input AND Mask Copyright (C) UNSW COMP9517 24T2W1 Image Processing Basics 43 Arithmetic and logical operations Useful logical operations include bitwise AND and OR Input Mask Input OR Mask Copyright (C) UNSW COMP9517 24T2W1 Image Processing Basics 44 Averaging Useful for example to reduce noise in images Assume the true noise-free image is ππ(π₯π₯, π¦π¦) and the actual observed images are ππππ π₯π₯, π¦π¦ = ππ π₯π₯, π¦π¦ + ππππ (π₯π₯, π¦π¦) for ππ = 1, β¦ , ππ, where the ππππ are zero-mean, independent and identically distributed (i.i.d.) noise images, then we have E ππππ (π₯π₯, π¦π¦) = ππ(π₯π₯, π¦π¦) and VAR ππππ (π₯π₯, π¦π¦) = VAR ππππ (π₯π₯, π¦π¦) = ππ 2 (π₯π₯, π¦π¦) ππ ππ ππ 1 1 1 β ππ Μ π₯π₯, π¦π¦ = ππππ (π₯π₯, π¦π¦) = ππ π₯π₯, π¦π¦ + ππππ (π₯π₯, π¦π¦) = ππ π₯π₯, π¦π¦ + ππππ (π₯π₯, π¦π¦) ππ ππ ππ ππ=1 ππ=1 ππ=1 ππ ππ 1 1 1 ππ 2 (π₯π₯, π¦π¦) β VAR ππππ (π₯π₯, π¦π¦) = 2 VAR ππππ (π₯π₯, π¦π¦) = 2 ππππ 2 (π₯π₯, π¦π¦) = ππ ππ ππ ππ ππ=1 ππ=1 Copyright (C) UNSW COMP9517 24T2W1 Image Processing Basics 45 Averaging Useful for example to reduce noise in images ππ = 1 ππ = 8 ππ = 16 ππ = 64 ππ = 128 ππ ππ/2.8 ππ/4 ππ/8 ππ/11.3 Copyright (C) UNSW COMP9517 24T2W1 Image Processing Basics 46 Further reading on discussed topics Sections 3.1-3.3 of Szeliski Chapter 3 of Gonzalez and Woods 2002 Acknowledgement Some images drawn from the mentioned resources Copyright (C) UNSW COMP9517 24T2W1 Image Processing Basics 47 Example exam question Which one of the following statements about intensity transformations is incorrect? A. Contrast stretching linearly maps intensities between two values to the full output range. B. Log transformation maps a narrow range of high intensities to a wider range of output values. C. Power transformation can map intensities similar to log and inverse log transformations. D. Piecewise linear transformations can achieve contrast stretching and intensity slicing. Copyright (C) UNSW COMP9517 24T2W1 Image Processing Basics 48