Object Recognition and Invariance

Object Recognition

• Involves assigning labels, from specific identification to broad categorization, to objects
• Requires processes like segmentation, context processing, and identification
• Machine-based systems have advanced rapidly, but human object recognition remains superior
• Machine-based systems are not well suited for low-pass images and are affected by texture bias

Object Invariance

• A challenge in object recognition is to quickly and accurately recognize objects despite variations
• Requires ignoring variance to categorize basic-level objects
• Objects can have subordinate-level variations, such as different chair types or fonts
• Any individual object can produce a range of images on the retina, requiring preserving identity
• Objects can be identified regardless of location, distance, angle, lighting, and context
• Object recognition remains consistent despite degradation like occlusion, distortion, and noise
• Visual system maps variable sensory inputs to stable neural activity for object identity

Advancements in Machine Learning

• Despite advancements, machine learning lacks human-like mechanisms
• This results in low-pass image inaccuracy and incorrect object classification due to texture bias
• Debate around invariance focuses on structural-description and view-based models

Ventral Visual Stream Function

• Vital for object perception and recognition, following a sequence of processing stages
• Visual data flows from the retina to the LGN in the thalamus, then through cortical areas V1, V2, and V4, ending in the inferotemporal cortex
• Progressive visual re-representations occur, with neurons along the processing hierarchy selectively responding to increasingly complex stimuli
• Activations at lower levels merge to form patterns representing object identification at the top of the hierarchy
• Receptive field size, stimulus selectivity, and space invariance increase gradually
• V1 and V2 neurons are sensitive to the orientation, length, and width of bar stimuli
• Neurons respond to specific orientations, increasing activity towards preferred orientations

V4 and Shape Processing

• V4 neurons selectively respond to oriented curves and edges, shaping contours
• Increase neural firing rate as preferred shape features appear within the receptive field
• Pasupathy & Connor (2002) studied population code for shape in V4 of macaque monkeys
• Individual V4 neurons are tuned to specific features, such as concave or convex curvatures
• Population activity in V4 encodes complete shapes
• Combining responses of V4 neurons with different tunings reconstructs shapes

Higher-Level Processing: IT and LOC

• Inferotemporal cortex (IT) and lateral occipital complex (LOC) operate above V4
• Neurons at these levels assemble meaningful, complex object representations from V4 activity
• Damage to IT/LOC causes visual object agnosia, impairing higher visual processes needed for recognition
• Low-level functions like color and motion remain intact
• IT/LOC is responsible for visual object recognition but not basic feature perception
• Gross et al. (1972) showed IT cortex neurons in macaques respond to complex stimuli like hands and faces
• IT cortex neurons demonstrate minimal responsiveness to simple stimuli
• Suggests that IT is critical to complex visual processing and object recognition
• Tanaka (1996) found that IT neurons are neuroanatomically organized by feature selectivity
• Adjacent neurons are selective for the same minimal features, and neurons exhibit maximal response to visually similar objects
• IT shows category-selective organization at a macro level
• Population response with fMRI reveals clusters of neurons with preferential responses to specific categories
• Chao et al. (1999) showed the fusiform face area responds to faces and animals
• Parahippocampal place area responds to houses and tools.

Models of Object Recognition - Information Processing

• Marr (1982) stated that understanding vision requires defining the problem and principles
• Explanations of visual perception and object recognition need a computational framework
• The framework should derive from the algorithms' operation and implementation
• Computational models treat the visual system as constructing a representation of the 3D structure of the distal scene

View-Dependent Model

• Uses a viewer-centered coordinate system to treat different views of an object as distinct
• Features are extracted from the image viewpoint and configurations of features are encoded
• Object recognition is holistic, matching whole images to image-like views stored in memory

Template Matching

• The simplest approach to object recognition
• Compares visual input to stored mental representations or images of objects
• Involves superimposing a stored pattern on the input and determining the degree of match
• Transformation of the mental image enables object recognition regardless of stimulus position, size, and orientation
• Matching requires prior normalization of the stimulus
• The approach fails when objects are distorted and requires an exact or near-exact match

Feature Matching

• Object recognition is based on comparing feature descriptors like edges and orientations
• Matches image features to model features
• In Selfridge's (1959) Pandemonium model for letter recognition, letters are identified via component features
• "Feature detector" demons identify low-level features and "shout" to cognitive demons
• Cognitive demons begin shouting, and a decision demon selects the best-fitting pattern basd on intensity
• A conceptual structure uses multiple layers with increasing complexity

Viewpoint Invariance Criticism

• A major criticism centers on the computational model's need to explain viewpoint invariance
• It is computationally impossible to possess representations of all possible objects/features from all possible viewpoints

Structural Description Model

• A viewpoint-independent model
• Emphasizes computing a canonical description that is independent of the viewer's vantage point.
• A single description of an object's spatial structure is stored in memory and is recognizable from all viewpoints
• Internal description of the object's structure is constructed from observed visual features
• This uses that to make a representation of the relationships of object parts to one another, independent of the viewpoint
• Generalizing across all viewpoints, sizes, is needed but should still allow discrimination

Marr's Computational Theory

• A sequence of successive conversions into increasingly complex representations of an image
• The input retinal image contains intensity and wavelength of light at each point
• The primal sketch is a conversion of intensity values into an edge image

Processing of Visual Features

• Computes zero-crossings to identify edges, curves, and boundaries
• 2.5D sketch represents the distance and orientation of visible surfaces and discontinuities in surface orientation
• Integrates edge images with depth cues from motion and surface texture.
• 3D model represents shapes organized using volumetric primitives
• Converts into an object-centered 3D model by identifying concave surfaces
• The image is divided into 3D primitive object elements, or "generalized cones"
• Objects are represented using hierarchical generalized cones that vary in length, orientation, and volume
• Objects are broken down into primitive sub-components and create a viewpoint-independent description
• Combinatorial power of geons is used to represent many objects with minimal memory load

Biederman's Recognition-by-Components (RBC) Theory

• Adapted the concept of generalized cones
• Visual system relies on non-accidental properties that are stable across viewpoint changes
• Non-accidental properties are features of edges in a 2D image that the visual system sees as evidence
• Properties are unlikely to result from viewpoint alignments
• Complex objects are broken down into 36 basic 3D shapes, or geons
• Recognition is based on unique geon combinations that create non-accidental properties
• Biederman's studies used obscuring geons

Contour Importance

• Contours in an image are critical for object recognition, so they aid the ability to distinguish object parts
• Deleting contours in midsections while preserving concavities means the object remains recognizable
• Deleting contours at concavities causes the object to be unrecognizable
• The position of line junctions may give diagnostic info about the identify

Further Points on Models of Object Recognition

• Primarily addresses basic-level recognition.
• Does not account for the recognition of objects at superordinate levels

Revisiting View-Dependent Model

• Bulthoff & Edelman (1992)
• Recognition is sensitive to viewpoint
• Participants were presented with wire-frame objects rotating in 3D space, and exposed to rotations of the objects
• Object Recognition Testing: object recognition for tested using new, unseen views, and rotations that were superior similar to views trained

Learning and Neuronal Response

• The scientists Logothetis et al. (1995) researched recording from object-selective neurones
• The research was done in the IT cortex of macaque
• It was done to determine encoding object degree, in viewpoint-invariant manner

Computer Generated Shapes Study Points

• The monkeys could classify and recognise computer objects
• The objects came and were present with a known object.
• The neurons responded selectively to learned views of computer 3D Objects
• With the angle distance of experienced viewpoint, this showed a response decline
• The majority that took part in the testing (object selective neurones), ended up as viewpoint-dependent
• Only a small number of nerve cells became viewpoint-invariant
• No neurons were preferentially selective for unfamiliar object views

Hierarchical Architectures with Feature Combinations

• Features become more complex as it moves up the hierarchy
• The structure shows two mechanisms
• Invariant to scale and invariant to feature

Structures Complexity Progression

• Neurons pool from responses (same Object ) through scaling, roation, translation
• Creation of view -invarient that is recognized
• Has the interpolation when is avoiding needs to store

Deep Learning Models

• Models come neural networks
• Multilayered
• The hierarchical models is alike in deep learning models
• Some layers amplify imporant feature that distinctions made easy
• The brain's IT neurons
• Research showed the objects were recognized from parts that simple the features (Tanaka, 1995)

More Insights on Neurons

• Ponce et al. (2019) used deep network to stimuli
• Neural maximized IT firing.
• The Neurons were selective for faces and in IT, as a responsde more attributes show up.
• The more complex feature attributes get coded with this.