Computer Vision: Motion Estimation

Computer Vision: Motion Estimation

• Introduction: Motion in images is analyzed through image sequences.
• Different natures of images: Images with the same dimensionality (e.g., 3D = 2D + time) have different characteristics, depending on the nature of the higher image dimensions.
• Applications: Motion-based recognition (e.g., human identification), automated surveillance (detecting suspicious activities), video indexing (automatic annotation), human-computer interaction (gesture recognition), traffic monitoring (gathering statistics), and vehicle navigation (path planning & obstacle avoidance).
• Scenarios: Analysis scenarios vary depending on whether the camera is still or moving, and if the scene's motion is coherent, or involves single or multiple moving objects.
• Topics:
  • Change detection: Detecting changes in images using image subtraction.
  • Sparse motion estimation: Estimating local displacements using template matching.
  • Dense motion estimation: Using optical flow to compute a dense motion vector field.

Change Detection

• Principle: Detecting moving objects against a constant background.
• Method: Subtracting the previous image (It-1) from the current image (It) reveals changes (motion).
• Edge Advance: Moving objects usually only advance a few pixels per frame.
• Algorithm steps:
  1. Acquire background (static): Capture a background image.
  2. Subtract: Subtract the background image from subsequent frames.
  3. Threshold & process: Process the difference image using thresholds to identify and isolate moving objects.

Image Subtraction Algorithm

• Input: Images (current and previous/reference frame) & intensity threshold.
• Output: Binary image (Iout), bounding boxes (B) of moving objects.
• Process:
  1. Compute the absolute difference between corresponding pixels in consecutive frames.
  2. Threshold the difference image. Pixel values that exceed the threshold qualify as changes (moving objects) and are marked as 1 in Iout, otherwise 0.
  3. Use connected components analysis (removing disconnected noise regions).
  4. Perform closing (smoothing small noise regions by merging them).
  5. Compute bounding boxes (B) of significant regions of changes (e.g., identified moving objects).

Sparse Motion Estimation

• Principle: Identifying corresponding points in consecutive images and estimating their displacement.
• Assumption: Intensities of interesting points and their neighbours remain roughly constant over time.
• Steps:
  1. Detect interesting points: Employ image filters (e.g., Canny, Hessian, Harris, SIFT, CNN-based) to identify points that are particularly indicative of motion.
  2. Search for corresponding points: Locate corresponding feature points (or their neighbourhoods) in succeeding frames within a specified search region to accommodate possible motion.

Sparse Motion Estimation: Procedure

• Detect interesting points (I,V,w,t) procedure: Traverse the frame, marking interesting points if the interest operator value exceeds a threshold.
• Interest Operator (I,r,c,w) procedure: Calculates the intensity variance in horizontal, vertical, and diagonal directions; the minimum of these variances is returned.

Sparse Motion Estimation: Search Corresponding Points

• Template Matching: Identify an interesting point in a frame and analyze its neighbourhood as a template.
• Search Region: Search for the best match in the subsequent frame within a defined region around the template (based on pre-defined constraints on motion limits).

Sparse Motion Estimation: Similarity Measures

• Methods (with goal):
  • Cross-correlation (to maximise): The degree of pixel intensity similarity between the template neighborhood and its match.
  • Sum of absolute differences (to minimize): Cumulative differences in pixel intensities from the template to its match.
  • Sum of squared differences (to minimize): Cumulative squared differences in intensities

Sparse Motion Estimation: Further Similarity Measure (Mutual Information)

• Principle: Maximize similarity using mutual information between subimages (to account for intensity differences):
• Compare subimages: Select sub-images for similarity checks to account for intensity variations.
• Intensity probabilities: Calculate individual probability distributions for intensity values in each sub-image.
• Joint intensity probability: Calculate the probability distribution of joint intensity in both sub-images.

Dense Motion Estimation

• Principle: Estimation of motion vectors for all pixels in the image.
• Assumptions: Light and distance properties remain constant, visual appearance doesn't change significantly within the small time interval (Δt).

Optical Flow Equation

• Assumption: Small neighbourhood undergoes a displacement vector (Δx, Δy) in time (Δt), maintaining intensity within the neighbourhood.
• Equation: f(x + Δx, y + Δy, t+Δt) = f(x,y,t). This equation encapsulates the premise underlying dense motion estimation (optical flow) calculations.

Optical Flow Computation

• Combining equations: Combining the Taylor series approximation and the optical flow equation yields a constraint equation
• ∇f • v = -ft : Equation for optical flow constraint.
• Spatial image gradient (vf): Calculates intensity changes across image frames, essential for the accuracy of subsequent computations.
• Temporal image derivative (ft): Calculates intensity changes in time (Δt), crucial for motion calculation.

Optical Flow Computation: Example (Lucas-Kanade)

• Approach: A method for calculating optical flow using a least squares approach over a neighborhood of pixels.

This quiz explores the intricacies of motion estimation in computer vision, focusing on the analysis of image sequences. It covers various applications such as automated surveillance, gesture recognition, and traffic monitoring, alongside methods like change detection and sparse motion estimation. Test your understanding of how motion is interpreted across different scenarios.