Introduction to Computer Vision

• Computer vision integrates AI, machine learning, cognitive science, neuroscience, image processing, computer graphics, and robotics.

Applications of Computer Vision

• Scene understanding enables movement in robots and autonomous driving.
• Optical Character Recognition (OCR) reads number plates, digits, Sudoku grids, and processes checks automatically.
• Face detection and recognition systems.
• Biometrics utilizes computer vision.
• Fine-grained recognition distinguishes between similar classes.
• Shape and motion capture is used for creating animated characters.
• Self-driving car technology relies on computer vision.
• 3D and 4D reconstruction creates models.
• Cross-modal image retrieval systems.

Semantic and Geometric Information

• Image enhancement improves photos through computational photography techniques.
• Super-resolution enhances image resolution.
• Low-light photography improves images in poor lighting conditions.
• Depth of field adjustments are used on cell phone cameras.
• Inpainting or image completion fills in missing parts of an image.
• Image synthesis generates new images, such as turning a horse into a zebra.
• Computer vision reconstructs 3D images from 2D images.

Challenges in Computer Vision

• Viewpoint variation changes the appearance of objects.
• Illumination changes affect how objects are perceived.
• Scale variations in object size.
• Intra-class variation within categories of objects.
• Motion blur.
• Background clutter.
• Occlusion where objects are partially hidden.
• Local ambiguity in interpreting image regions.

Cues in Computer Vision

• Depth cues such as linearity.
• Shape and lighting cues such as shading.
• Grouping cues based on color and texture similarity.
• Shape cues based on texture gradient.

Further Challenges

• Learning from fewer labels with low-shot, semi-, self-, and weakly supervised learning, and continual learning.
• Domain adaptation with the same cameras but different images.
• Autonomous driving technologies.
• Network compression and pruning.
• Fine-grained image analysis.
• Face verification/recognition systems.
• Image search and retrieval systems.
• Style transfer and image synthesis.

Images as Data

• Images represented as a grid (matrix) of intensity values with two dimensions.
• Grayscale images have X and Y values representing intensity.
• Color images have one grayscale image per RGB channel or a 3D vector per point (X, Y).

Image Transformations

• Image transformations apply operators to an image.

Image Filtering

• Image processing/signal processing through filters on data arrays.
• Filters extract edges or corners of the image.
• Filters reduce noise in images.
• Filters sit on pixels, with the center pixel as the result.

Cross-Correlation

• Cross-correlation involves applying a filter (H) to an image (F), multiplying from one corner across the image.

Convolution

• Applying a filter (H) to an image (F), passing through the image, then changing the filter orientation.
• Convolution is commutative and associative.
• Values set to process an image through a filter extract features or information.
• Symmetric filters yield the same result for convolution and cross-correlation.

Linear Filtering

• Linear filtering replaces each pixel with a weighted sum of its neighbors.

Mean Filtering/Moving Average

• Computing the average value across numbers in the filter.
• Smooths out and blends edges, reducing sharp contrast.
• The center has high values and the surrounding has lower values.
• Padding maintains image size and stride determines filter movement.

Sharpening Linear Filter

• A doubling filter and a main filter sharpen the edges of an image.

Ringing Artifacts

• Ringing artifacts occur when an image with edges has a box filter applied.

Gaussian Filter

• Image blurs with increased sigma value.
• More preferred than mean filtering when the image has many edges.

Median Filter

• Used to remove salt and pepper noise or speckle noise, a non-linear operation.
• It gets rid of outliers.

Thresholding

• Non-linear filtering sets a threshold.
• Global threshold applies to the entire image.
• Adaptive threshold sets values locally for parts of the image.