5 Questions
0 Views

# Computer Vision - Optical Flow

Created by
@TenderAstatine

### What is the goal of finding optical flow for each pixel in an image?

To find a velocity vector for each pixel indicating how quickly and in which direction it is moving across the image.

True

### In Lucas-Kanade method, the overconstrained linear system can be solved using the __________ method.

<p>Least-Squares</p> Signup and view all the answers

### How does multi-scale flow estimation improve optical flow estimation?

<p>It runs the Lucas-Kanade algorithm at different scales to capture motion details.</p> Signup and view all the answers

## Study Notes

### Optical Flow

• Optical flow is the pattern of motion of objects, surfaces, and edges in a visual scene
• It is used to understand the motion of the camera and objects in a scene

### Scene Interpretation

• Given a video sequence, we can find the motion of the camera and objects by:
• Recovering camera ego-motion
• Motion segmentation
• Structure from motion

### Applications

• Optical flow has applications in:
• Recovering camera ego-motion
• Motion segmentation
• Structure from motion
• Multi-body segmentation
• Recognizing the type of motion (e.g., walking, running, etc.)

### Motion Field and Optical Flow

• Motion field: the real-world 3D motion
• Optical flow field: the projection of the motion field onto a 2D image

### Examples of Motion Fields

• Forward motion
• Rotation
• Horizontal translation
• Closer objects appear to move faster

### When Does It Break?

• The optical flow field can break when:
• The screen is stationary, but objects generate motion
• Non-rigid objects change shape
• The light source changes

### Methods for Determining Optical Flow

• Phase correlation methods
• Block-based methods
• Differential methods, including:
• Horn-Schunck method
• Buxton-Buxton method
• Black-Jepson method

### Estimating Optical Flow

• Assume the image intensity is constant
• Use the brightness constancy equation: I(x, y, t) = I(x + dx, y + dy, t + dt)
• First-order Taylor expansion: I(x, y, t) + Ix * dx + Iy * dy + It * dt = 0

### Brightness Constancy Equation

• Simplify the notation: Ix * u + Iy * v = -It

### Problem I: One Equation, Two Unknowns

• The brightness constancy equation has one equation and two unknowns (u and v)

### Problem II: The Aperture Problem

• For points on a line of fixed intensity, we can only recover the normal flow
• We need additional constraints to solve the aperture problem

• Use local information to solve the aperture problem
• Lucas-Kanade method (1984): assume constant (u, v) in a small neighborhood

### Least-Squares Method

• Solve the overconstrained linear system using the least-squares method
• Minimize the sum of squares of "errors" between the right- and left-hand sides of the equations

• Edge regions: A^T * A becomes singular
• Homogeneous regions: A^T * A is close to zero
• Textured regions: two high eigenvalues

### Other Break-Downs

• Brightness constancy is not satisfied
• A point does not move like its neighbors
• The motion is not small (Taylor expansion doesn't hold)

### Multi-Scale Flow Estimation

• Use multi-scale estimation to handle large motions
• Run Lucas-Kanade at multiple scales and warp and upsample the results

### Example: Motion-Based Segmentation

• Input: a video sequence
• Segmentation result: separate objects moving in different ways

### Affine Motion

• For panning camera or planar surfaces, use affine motion: u = p1 + p2 * x + p3 * y, v = p4 + p5 * x + p6 * y
• Only 6 parameters to solve for, resulting in better results

## Studying That Suits You

Use AI to generate personalized quizzes and flashcards to suit your learning preferences.

## Description

This quiz covers optical flow in computer vision, understanding motion in video sequences, and scene interpretation. It involves tracking pixel movements between images and analyzing camera/object motion.

## More Quizzes Like This

5 questions
18 questions
Use Quizgecko on...
Browser
Information:
Success:
Error: