4. Motion copy.docx

CRITICAL READING: CORNELL NOTES Motion Name: Date: 8 August 2023 Section: Lecture 4 Period: Questions/Main Ideas/Vocabulary Notes/Answers/Definitions/Examples/Sentences Motion Movement of an object across your visual field translates to movement in your retinal image. Figure-ground...

CRITICAL READING: CORNELL NOTES Motion Name: Date: 8 August 2023 Section: Lecture 4 Period: Questions/Main Ideas/Vocabulary Notes/Answers/Definitions/Examples/Sentences Motion Movement of an object across your visual field translates to movement in your retinal image. Figure-ground segregation. Extraction of three-dimensional structure (relative movement). Visual guidance of action. Social communication. Figure-Ground Segregation Shapes and objects that are camouflaged while static, and therefore invisible, can be segregated from their background as soon as they move on the basis of the relative motion. Prey animals such as small lizards and rodents move in short, rapid bursts of activity to minimise the time that they are visible to predators in this way. Extraction of Three-Dimensional Structure When any solid object moves, the retinal images of its various parts move relative to each other. For example, when you view a rotating glove, surface markings near the equator move across your field of view more rapidly than markings near the poles. There is also highly structured variation in direction. The changes in speed and direction carry quite detailed information about the 3D structure and distance of the shape, which is similar in certain respects to the depth information contained in the static texture gradients. Visual Guidance of Action The pattern of optic flow can be used to estimate the speed and direction of self-motion. For example, as you drive down a road, focusing on the horizon, image details from road markings, signposts, and pedestrians appear at the horizons and move through your field of view to create and expanding flow field. The rate of flow provides information on your speed, and the focus of expansion indicates your heading direction. Social Communication Movement is a significant component of human nonverbal communication in terms of gestures, dynamic facial expressions and whole body movements. Lips movements are important for verbal communication. How Might we Detect Motion? An object moving in the real world will also move across your retina. Sequential activation of adjacent receptors can be used to signal motion. Illumination The figure depicts the retinal image of a running figure seen in silhouette, at two slightly different times. The figure is darker than the background, so as it moves into previously empty retinal space the receptors in those regions will signal a decrease in retinal illumination; similarly receptors in regions vacated by the figure will signal an increase in retinal illumination as it returns to background level. The changes in illumination picked up by individual receptors tell us nothing about the movement of the figure. Movement in the opposite direction would also produce some increases and some decreases in illumination. Simple Motion Detection Circuit Retinal movement A and B are receptor cells. X is a comparator neuron (input A, B). Comparator neuron X performs motion detection from two spatially separated receptor cells, with a temporal delay between them. Distance between A, B = ∆s. Signal from A to X > B to X = ∆t. The comparator multiplies signals from A and B. Largest response when A, B signals arrive at the same time. Weaker response when A, B signals arrive separately. Direction Selectivity We can examine the output of the comparator during the passage of the image of a car across the retina from left to right. At time 2 the car’s image falls on receptor A, evoking a large response (10 units of activity) that begins traveling toward the comparator. The response will take one time interval to reach the comparator. By the next time interval (time 3) the image has reached the second receptor, B. Response at receptor A drops back to zero, but the high response at receptor B (10 units of activity) reaches the comparator immediately. Thus, at time 3 the two responses from A and B arrive together at the comparator, creating a very large response (100)—motion is detected. The figure below shows the pattern of responses when the car moves in the opposite direction. In this case the two receptors’ signals arrive at the comparator at different times. At time 1 receptor B’s response travels immediately to the comparator C, provoking a low response. At time 2 the image of the car has passed on to receptor A. The resulting response does not arrive at the comparator until time 3, resulting in no response from the comparator. The neural circuit is thus selectively responsive to motion from left to right. A motion detector selectively responsive to motion from right to left can be made simply by rewiring the circuit so that the temporal delay is imposed on the line from receptor B to the comparator instead of on the line from receptor A. Velocity Selectivity Motion detectors of this kind are very sensitive to the properties of the motion stimulus. In particular, the velocity of the stimulus has to be such that the time taken for it to traverse the distance from A to B matches the difference in transmission speed in the two arms of the circuit. Only then will the signals from A and B arrive at C together, even for the preferred direction. If there is a mismatch because the stimulus moves too slowly or too rapidly, the detector may fail to signal motion at all—it is velocity sensitive. In order to deal with a range of image velocities, the visual system would require a population of detectors that vary in terms of the transmission speeds and/or spatial offsets built in to their circuits. Barlow & Levick (1965) Cells in the rabbit retina are directionally selective. Directional tuning. Suggests a building block for motion models. Reichardt Model (1961) Contains banks of temporal and spatial filters. Two mirror symmetrical units. Comparison of these outputs used to detect motion. Opponent energy + values signal movement in one direction, - values the other. “Delay and compare” model. Limitations of a Simple Motion Detector First order motion A motion stimulus containing shapes defined by variations in luminance between the object and the background. Discrete Correspondence The object seen at time 1 is the same object seen at time 2. Second Order Motion A motion stimulus consisting of shapes defined by textures, without corresponding changes in luminance. The stimulus on the previous slides consists of random black and white squares. One column of the grid reverses contrast on each slide. We perceive rightward motion > even though the same feature doesn’t move. Simple motion detectors don’t account for the perception of second order motion. Motion Energy Model Motion detection in the primate visual system is based on the same principles but is located in the visual cortex. A and B aren’t individual photoreceptors but are immediate neurons (LGN). Proposed at the same time by: Adelson & Bergen (1985) Burr, Ross and Marrone (1986) Rectangles on the figure: Receptive fields. The angle of the stimulus = velocity. Spatial Orientation of Receptive Field for V1 Simple Cells Spatio-Temporal Orientation of Receptive Field in Energy Model Reichardt vs Energy Model Energy model has similar banks of spatial and temporal filters as Reichardt model. But outputs from quadrature pairs are squared and summed to calculate motion energy. The end result is very similar, but there are benefits to the energy model. Benefits of an Energy Model Specificity Motion response is to a particular space, time and spatial frequency. No need for discrete correspondence Better chance of detecting motion. EM doesn’t solve correspondence but bypasses the issue entirely. More flexible definition of a feature Ability to detect second-order motion in random noise patterns. Motion After Effects Opponent coding: Cells tuned for downwards motion are balanced against those for upwards motion. Demo: Fatiguing “up” cells means that “down” cells become temporarily dominant. Leading to the perception of motion on a still image. Interocular transfer: Motion detection must be in the cortex. Middle Temporal Area (MT) MT neurons are selective for motion direction. Receives input from directionally selective V1 cells. But while V1 cells respond to orientation moving in direction, MT responds to any pattern. MT neurons are velocity selective (speed and direction). Activation correlated with motion perception. Acquired Motion Deficits Zihl et al. (1983) report on patient ‘LM’. LM suffered a stroke (V5/MT): Retained normal visual acuity, colour, etc. Retained normal object/face recognition. But lost the ability to perceive motion. Moving cars Walking people Moving mouths and facial expressions Pouring liquid Local vs Global Features Local feature: individual elements of the shape. Global feature: describes the entire shape. Local receptive fields don’t have enough information to disambiguate direction of the object in motion. V1 codes direction of local features. A cell in MT pools input from many V1 cells, creating the percept of right horizontal motion. The Aperture Problem The direction in which the pattern appears to be moving depends on the orientation of the aperture. The Aperture Problem -Explained Motion of a repetitive e pattern in perceived as moving in an ambiguous direction because the pattern is featureless. Motion in any direction (a, b or c) isn’t perceived as its true direction. Solving the Aperture Problem Orientation information can be used to disambiguate edges (line endings). Changing the orientation of the aperture changes perceived direction of motion. Pooling orientation (or form) and motion information – which happens over many spatial scales – aids disambiguation of motion direction. Motion vs Form Processing Dorsal stream (‘Where’): V1 > V2 > V3 > MT > Parietal lobe Motion processing “Motion” Ventral stream (‘What’): V1 > V2 > V4 > Inferior temporal Object processing “Form” Motion Blur Integrating form into motion. Cells in V1 integrate over ~100ms. Results in a “blur” along the axis of motion for moving objects. But humans don’t see blue suggests that motion deblurring mechanisms exist. Motion Streaks Motion blur effectively creates an orientation signal along the axis of motion. The form information can be used to disambiguate direction of motion. Geisler (1999): Method: Target detection among noise. Result: The target was harder to detect when the noise was parallel to motion interference. Using Motion Streaks to Detect Motion Direction Geisler (1999): Combined input from orientation and direction selective neurons (or populations of neurons) in V1. Integration time creates blur. Gives the ability to disambiguate motion. Biological Motion Johansson (1973) demonstrated that people are able to recognise people walking, even when there are only points of light on the joints. Example of motion defining form. Further evidence of integration of dorsal and ventral processing streams. Likely in the superior temporal sulcus (STS).