Image Processing and Projective Transformations

Course Information

• Course title: Introduction to Computer Vision
• Course code: CPS834/CPS8307
• Instructor: Dr. Omar Falou
• University: Toronto Metropolitan University
• Semester: Fall 2024

Image Transformations and Warping

• Image transformations modify an image's geometric structure.
• Image warping changes an image's coordinate domain, not its range.
• Transformations include translation, similarity, projective, Euclidean, and affine.
• Reading material: Szeliski, Chapter 3.6

Image Alignment

• Images may not perfectly align due to differences in perspective or camera position.
• Geometric relationships analyze the spatial connections between images.

Image Warping

• Image filtering manipulates the intensity range of an image.
• Image warping changes the coordinate domain of an image.
• Mathematical formula examples: g(x) = h(f(x)) and g(x) = f(h(x))

Parametric (Global) Warping

• Examples include translation, rotation, and aspect ratios.
• Parametric transformations can be described using a few parameters.
• Transformation T is a coordinate-changing machine, where p' = T(p).
• Linear transformations (can be represented by a 2x2 matrix) are used, where p' = Tp.

Common Linear Transformations

• Uniform scaling (by a factor 's').
• Rotation (by an angle θ about the origin).
• The inverse of a rotation is its transpose. Mathematical formula example: R= [cos θ -sin θ] [sin θ cos θ]

2x2 Matrices

• Some transformations (mirrors across axes or lines) can be represented with 2x2 matrices.
• 2D translation cannot be represented using a 2x2 matrix.

All 2D Linear Transformations

• Linear transformations include scale, rotation, shear, and mirror.
• They map the origin to the origin.
• They map lines to lines, and parallel lines remain parallel.
• They preserve ratios.
• They are closed under composition.

Homogeneous Coordinates

• Representing points using homogeneous coordinates helps incorporate translation into transformation matrices.
• (x, y) becomes (x, y, w).
• Conversion from homogeneous coordinates: (x/w, y/w).

Translation

• In homogeneous coordinates, translation can be represented using a 3x3 matrix.
• Example matrix: T = [1 0 tx] [0 1 ty] [0 0 1 ]

Affine Transformations

• Affine transformations are combinations of linear and translation transformations.
• The origin does not necessarily map to the origin.
• Lines map to lines and parallel lines remain parallel.
• Ratios are preserved.
• Closed under composition.

Projective Transformations (Homographies)

• A projective transformation maps points in 2D space to another plane. It can be represented by a 3x3 matrix.
• Example matrix: H = [a b c] [d e f] [g h 1]

Points at Infinity

• Projective transformations involve points at infinity

Image Warping with Homographies

• These transformations involve mapping points in an image to another location.

Properties of Projective Transformations

• Origin does not necessarily map to origin.
• Lines map to lines.
• Parallel lines do not necessarily remain parallel.
• Ratios are not preserved.
• Projective transformations are closed under composition.

2D Image Transformations

• Various types of 2D transformations (translation, rigid transformations, similarity, affine, projective) exist.
• Each transformation has a specific matrix that represents it and the number of degrees of freedom (e.g, 6 degrees of freedom for affine transformations).

Implementing Image Warping

• Forward warping involves sending each pixel to its corresponding location in the output image.
• Inverse warping involves getting each pixel in the output image by transforming back to the location in the source image.

Forward Warping

• Pixels need to be interpolated if mapping falls between pixels.

Inverse Warping

• Interpolated values from the source image determine pixel values.

Interpolation

• Interpolation is about estimating values between sampled points.
• Common filters for interpolation are nearest neighbor, bilinear, bicubic, and sinc filters.

Image Resampling and Interpolation

• Includes resampling and interpolating between sample points
• Key for upscaling or downscaling images
• Nearest neighbor interpolation, bilinear interpolation, and bicubic interpolation are frequent methods.

Image Scaling

• Techniques to increase or decrease image dimensions proportionally.
• Common method is image sub-sampling where pixels are skipped.

Image Sub-sampling

• Throwing away pixels to create images of reduced dimensions (e.g, 1/2 or 1/8 size).
• Leads to image blurring or "crushing" of content, known as aliasing.
• Gaussian prefiltering can reduce aliasing by blurring the image before sub-sampling

Aliasing

• Occurs when sampling rate is too low for image details.
• Leads to distortions (like the "wagon wheel" artifact). -Nyquist rate is the minimum sampling rate to avoid aliasing.

Wagon-wheel effect

• It's a visual artifact that appears when the scene or object rotates in a video, but the sampling rate doesn't keep up with the rotation.
• Leads to the appearance that it's moving in the opposite direction than expected.
• The problem can be prevented by higher sample rates.

Temporal Aliasing

• The effect that occurs in video when the frame rate isn't high enough to keep up with an object's motion.
• Causes the object to appear to be moving slower or backwards.

Gaussian Pre-filtering (Antialiasing)

• Blurring the image using a Gaussian filter before downsampling.
• Reduces aliasing effects and preserves image detail more accurately.
• Enables generation of proper downsampled images while resolving possible artifacts.

Gaussian pyramids

• A set of images in which each successive level has lower resolution.
• Created by blurring and sub-sampling images repeatedly.
• Useful for image processing tasks that involve multiple resolution levels.

Upsampling

• Increasing image dimensions
• Typically involves interpolating values for new pixels.
• Nearest neighbor interpolation and other methods like Bicubic interpolation can be used.

Image Interpolation

• Estimating values for new pixels between sampled points in upsampling.
• Important for generating images at higher resolutions.
• Reconstruction filters are useful for estimating values at any resolution.

Raster-to-vector Graphics

• Technique to convert pixel-based graphics to vector graphics (lines and curves, not pixels). Image tracing and vector magic are examples of this transformation.
• Useful for maintaining image quality when scaling and manipulating images.
• Good for image compression and scalability when scaling the images.

Depixelating Pixel Art

• Methods to improve the quality of pixel-based art.
• Aims at increasing the detail within pixelated art.

Modern Methods

• Modern techniques for image upsampling, including algorithms like RAISR.
• Advanced techniques for pixelated image resolution enhancement and better quality image generation.

Super-resolution with Multiple Images

• Utilizing multiple images of a scene with slight shifts to enhance resolution.
• Effectively used in some cellphone cameras.

Google Pixel 3 Super Resolution Zoom

• Cell phone technique for enhancing resolution while keeping the quality of the image.
• Improves the quality of images at higher resolutions while addressing problems associated with hand tremors and other artifacts

Edge Detection

• Identifying and highlighting boundaries or transitions in image intensities.
• Extracting vital image characteristics and details.

Origin of Edges

• Factors behind edges, including surface normal discontinuities, depth discontinuities, surface color discontinuities, and illumination discontinuities.

Images as Functions...

• Representation of images as functions showing how abrupt changes in intensity correspond to edges in images.
• Useful for analyzing and interpreting images.

Characterizing Edges

• Edges are characterized as rapid changes in image intensity functions.

Image Derivatives

• Using differential calculus to extract edge information from images.
• Finite difference approximations are methods to estimate derivatives from image data.

Image Gradient

• The gradient shows the direction and magnitude of the maximum rate of increase in image intensity.
• The magnitude corresponds to the edge strength and its direction represents the edge orientation.

Effects of Noise

• Noisy images have random variations in image values causing artifacts and affecting the accuracy in edge detection. -Methods for removing noise are needed before edge detection or characterization procedures.

Solution: Smoothing First

• Applying a smoothing filter to an image before edge detection reduces noise effects.

Associative Property of Convolution

• Convolution can be used to smooth images, estimate derivatives or edges and it's associative.

The 1D Gaussian and its Derivatives

• Gaussian distributions and their derivatives are used for image smoothing and edge detection.
• Gaussian distribution can be used to smooth the image values and enhance edge characteristics.

2D Edge Detection Filters

• 2D Gaussian or Laplacian filters are used to detect edges at different resolutions, and to highlight important image features.

The Sobel Operator

• An approximation of the derivative of Gaussian used to find edges in images by using partial derivatives on each dimension.
• It's often used to locate and/or delineate edges.

Canny Edge Detector

• Detecting edges in images to reveal boundaries or significant changes in image characteristics.
• A commonly employed method in computer vision.
• The parameters include smoothing, gradient magnitude, and thresholding rules to refine edge locations.

Test your knowledge on various aspects of image processing, focusing on subsampling, interpolation methods, and projective transformations. This quiz covers fundamental concepts like blurring, warping, and the effects of transformations on images. Challenge yourself with questions that probe your understanding of these advanced techniques in image manipulation.