Image Processing and MRI Fundamentals

AIS412 Lecture 7: Image Registration

• Lecture material sourced from Carnegie Mellon University's Computer Vision course and Stanford University's CNN for Visual Recognition course.
• Image registration involves finding the mapping function (T) to align homologous pixels in two images.
• A pixel in the first image (p) is mapped to a homologous pixel in the second image (q) using a transformation function (T(p; θ), where θ represents the unknown parameters of the mapping).
• Example mapping function:
  • T(p;θ) = (sx + tx, sy + ty) where s = scaling factor, tx = horizontal translation, and ty = vertical translation
• Problems in image registration:
  • Determining the mapping function (T)
  • Identifying unknown mapping parameters (θ)
  • Defining correspondences between pixels in the two images (p, q)

Applications

• Multimodal Integration:
  • Used when two or more sensors view the same region/volume.
  • Different sensors (e.g., range images vs. intensity images or CT volumes vs. fluoro images) provide different information about the same subject, requiring registration.
  • Some sensors provide redundant information from the same modality, so registration is not necessary.
• Example: MR-CT Brain Registration:
  • MRI measures water content; CT measures x-ray absorption.
  • Bone appears brightest in CT, darkest in MR.
  • Both images are 3D volumes.
• Image Mosaics:
  • Used for many overlapping images to construct a complete view as a mosaic (e.g., stitch multiple images together).
  • Requires limited camera viewpoint and simple geometric constraints (e.g., rotation about the optical center of simple geometry like a plane or quadratic surface).
• Building 3D Models:
  • Range scanners produce a set of 3D measurements at each pixel location.
  • Necessary to register multiple range images and texture map to create a 3D model.
  • Common application in reverse engineering and digital architecture/archaeology.
• Change Detection:
  • Used when images from different times are compared.
  • Registration allows differences between images to be identified, potentially indicative of change.
  • Sometimes difficulties when determining whether change exists.
• Multi-subject Registration/Object Recognition/Industrial Inspection:
  • Multi-subject registration creates organ variation atlases, used to detect abnormal variations.
  • Object recognition involves aligning an object model with an image of an unknown object. Industrial inspection involves comparing a CAD model to an instance of a part.

Steps Toward Image Registration/Problem Definition

• Analyse the images
• Define the appropriate image primitives (geometric and intensity)
• Determine the transformation models
• Design an initialisation technique
• Develop constraints and error metrics on transformation estimates
• Design a minimization algorithm
• Develop a convergence criteria

Pervasive Questions:

• Identify intensity information or image structures in a given set of images that might be consistent between the images.
• Analyse the geometric relationship between the image coordinate systems.
• Establish prior information that confines the possible transformation domains.

Spatial Transformations

• Rigid: Preserves lengths and angles (translation, rotation).
• Affine: Preserves parallel lines, but not lengths and angles (scaling, shear, rotation, translation)
• Projective: Doesn't preserve parallelism, lengths or angles
• Perspective: A specific type of projective transformation related to planar homography
• Global Polynomial (Spline): used for complex transformations

Similarity Metrics in Registration

• Absolute difference: Simple measure of difference between corresponding pixel values in images.
• SSD (Sum of Squared Differences): Calculates the sum of squared differences between corresponding pixel values in images, providing a more comprehensive comparison than absolute difference.
• Correlation Coefficient: Employs the correlation between corresponding pixel values in images to provide a measure of similarity/alignment.
• Mutual Information/Normalized Mutual Information: Used to determine the relationship between corresponding pixel intensity values in images.

Search Strategy

• Powell's direction set method, downhill simplex method, dynamic programming, relaxation matching, hierarchical techniques

Multi-modality Brain Image Registration

• Intensity-based 3D/3D rigid transformation (6 degrees of freedom).
• Maximisation of Normalized Mutual Information.
• Simplex downhill/Multi-resolution

Mutual Information as a Similarity Measure

• Mutual information is used to quantify statistical dependence between corresponding voxel intensities in aligned images.

Normalized Mutual Information

• Extends mutual information to account for sensitivity to overlap changes

Image Coordinate Transformations

• Image features (dimension, voxel size, slice spacing, gantry tilt, orientation) may differ due to acquisition from distinct modalities.
• Coordinate transforms are used to convert voxel units (column, row, slice spacing) to millimeter units, with the origin centred within the volume of the image.

Target Image & Template Image

• Different images for comparison/analysis; images to be transformed is termed the "target" or "moving" image while the second image is the "template" or "fixed" image.

Interpolation Methods

• Nearest Neighbor, Tri-linear, Partial-Volume, Higher-order partial-volume interpolation.

Optimization Methods used in Image Registration

• Powell's Direction Set method/Downhill Simplex method.

Further topics in the lectures (from other pages):

• Bag-of-Words (BoW) for image classification (a technique that learns representations from visual features).
• Types of Challenges for Image registration: Variable Viewpoint, Variable Illumination, Scale, Deformation, Occlusion, Background Clutter, Intra-class Variations.
• Feature extraction techniques: Regular grid, Interest point detectors/Other methods (SIFT).
• Methods for Learning Dictionaries (K-means clustering).

This quiz covers key concepts in image processing and magnetic resonance imaging (MRI). It explores topics ranging from image registration challenges to the bag-of-words model in classification. Test your knowledge on optimization methods and 3D modeling techniques.