Chapter 6.docx

*Groucho Marx* CHAPTER Object Recognition ================== - What processes lead to the recognition of a coherent object? - How is information about objects organized in the brain? - Does the brain recognize all types of objects using the same processes? Is there something special a...

*Groucho Marx* CHAPTER Object Recognition ================== - What processes lead to the recognition of a coherent object? - How is information about objects organized in the brain? - Does the brain recognize all types of objects using the same processes? Is there something special about recognizing faces? ### Computational Problems in Object Recognition 1. *Use terms precisely*. At a fundamental level, cases like that of patient G.S. force researchers to be precise when using terms like *perceive* or *recognize*. G.S. could see the pictures, yet he failed to perceive or recognize them. Distinctions like these constitute a core issue in cognitive neuroscience, highlighting the limita- tions of the language used in everyday descriptions of thinking. Such distinctions are relevant in this chap- ter, and they will reappear when we turn to problems of attention and memory in Chapters 7 and 9. 2. *Object perception is unified.* Although our sensory systems use a divide-and-conquer strategy (as we saw in Chapter 5), our perception of objects is uni- fied. Features like color and motion are processed along distinct neural pathways. Perception, however, requires more than simply perceiving the features of objects. For instance, when gazing at the northern coastline of San Francisco (**Figure 6.1**), we do not see just blurs of color floating among a sea of various shapes. Instead, our percepts are of the deep-blue water of the bay, the peaked towers of the Golden Gate Bridge, and the silver skyscrapers of the city. 3. *Perceptual capabilities are enormously flexible and robust*. The city vista looks the same whether we view it with both eyes or with only the left or the right eye. Changing our position on a San Francisco hill- side may reveal the expanse of Golden Gate Park or present a view in which a building occludes most of the park, yet despite this variation in sensory input, we readily recognize that we are looking at the same city. Indeed, the percept remains stable even if we stand on our head and the retinal image is inverted. 4. *The product of perception is intimately interwoven with memory*. Object recognition is more than linking fea- tures to form a coherent whole; that whole triggers memories. Those of us who have spent many hours roaming the hills around San Francisco Bay recognize that the pictures in Figure 6.1 were taken from the Marin headlands just north of the city. Even if you have never been to San Francisco, when you look at these pictures, there is interplay between perception and memory. Indeed, part of memory retrieval is recognizing that things belong to certain categories. For the traveler arriving from Australia, the first view 1. *Viewing position.* Sensory information depends highly on your viewpoint, which changes not only as you  2. *Illumination.* While the visible parts of an object may differ depending on how light hits it and where 3. *Context*. Objects are rarely seen in isolation. People see objects surrounded by other objects and against varied backgrounds. Yet we have no trouble sepa- rating a dog from other objects on a crowded city street, even when the dog is partially obstructed by pedestrians, trees, and hydrants. Our perceptual system quickly partitions the scene into components. TAKE-HOME MESSAGES - Sensation, perception, and recognition are distinct phenomena. - Object constancy is the ability to recognize objects in countless situations, despite variation in the physical stimulus. ### Multiple Pathways for Visual Perception The "What" and "Where" Pathways Representational Differences Between the Dorsal and Ventral Streams \ Perception for Identification Versus Perception for Action  \ TAKE-HOME MESSAGES - The ventral stream, or occipitotemporal pathway, is specialized for object perception and recognition. This is often referred to as the "what" pathway. It focuses on vision for *recognition*. - The dorsal stream, or occipitoparietal pathway, is special- ized for spatial perception and is often referred to as - Neurons in the parietal lobe have large, nonselective receptive fields that include cells representing both the fovea and the periphery. Neurons in the temporal lobe have large receptive fields that are much more selective and always represent foveal information. - Patients with selective lesions of the ventral pathway may have severe problems in consciously identifying objects, yet they can use the visual information to guide coordinated movement. Thus we see that visual informa- tion is used for a variety of purposes. - Patients with optic ataxia can recognize objects but cannot use visual information to guide action. Optic ataxia is associated with lesions of the parietal cortex. ### Seeing Shapes and Perceiving Objects Shape Encoding **Feature** **Sample stimuli** **Familiar** **Novel** **Scrambled** a. Stimuli for the three conditions and the mental operations required in each condition. Novel objects are hypothesized to engage processes involved in perception even when verbal labels do not exist. b. When familiar and novel objects were viewed, activation was greater in the occipitotemporal cortex, shown here in a horizontal slice, than when scrambled stimuli with no recognizable object shape were viewed. OFM From Shapes to Objects Left Grandmother Cells and Ensemble Coding Exploiting the Computational Power of Neural Networks Output Hidden Input **b** Deep feedforward Top-Down Effects Animals **c** 100 80 60 40 20 **a** LH OFC 130 ms **b** 1.0 0.8 0.6 0.4 0.2 Mind Reading: Decoding and Encoding Brain Signals 00 100 200 300 Time from stimulus onset (ms) BOLD response Stimulus a. Receptive-field encoding model of voxels in human V1. After the BOLD response to thousands of images is recorded, the receptive field of each voxel in V1 can be  30 60 a. Voxel AV-8592 (left parahippocampus) **b** Voxel AV-19987 (right precuneus)  a. Original images  b. Reconstructions a. Representative natural images (out of a nearly infinite set) that were presented to the model. b. The reconstructed images, based on a hybrid model of multivoxel responses across multiple visual areas. The model was developed by measurement of the BOLD response to a limited set of stimuli. Time to awakening (s) Time to awakening (s) TAKE-HOME MESSAGES - People perceive an object as a unified whole, not as an assemblage of bundles of features such as color, shape, and texture. - The lateral occipital cortex is critical for recognition of an object's shape. - The term *grandmother cell* has been coined to convey the notion that recognition arises from the activation of neurons that are finely tuned to specific stimuli. Ensem- ble theories, in contrast, hypothesize that recognition is the result of the collective activation of many neurons. - Recent advances in artificial intelligence have shown how multilayered neural networks with massive connectivity may be ideal for extracting regularities in the environment---a key computation for recognition and categorization. - Object recognition, especially of ambiguous stimuli, ap- pears to be enhanced by top-down processes, including information provided from the frontal cortex based on a fast but crude analysis of the visual input. - Encoding models are used to predict the physiological response, such as the BOLD response, to a stimulus. Decoding models are used in the reverse manner, predict- ing the stimulus (or mental state) from a physiological re- sponse such as the BOLD activity across a set of voxels. 4. ### Specificity of Object Recognition in Higher Visual Areas Is Face Processing Special? A B C D E  F G H I J 70 60 50 40 30 20 10 0 --10 70 60 50 40 30 20 10 0 --10  20 40 60 80 100 120 140 160 180 20 40 60 80 100 120 1 0 --1 0.6 0.4 0.2 0 0.6 0.4 0.2 0 Faces \> Objects Intact faces \> Scrambled faces a. Bilateral activation in the fusiform gyrus was observed with fMRI when participants viewed collages of faces and random patterns, as compared with collages of only random patterns. Note that, according to neuroradiological conventions, the right hemisphere is on the left. **(b)** In another fMRI study, partici- pants viewed alternating blocks of stimuli. In one scanning run (top), the stimuli alternated between faces (F) and objects (O); in another run (bottom), they alternated between intact (I) and scrambled (S) faces. The right column shows the BOLD signal in the fusiform face area during the scanning run for the various stimuli. In each interval, the stimuli were drawn from the different sets, and these stimuli were separated by short intervals of fixation only. The BOLD signal was much stronger during intervals in which faces (versus objects) or intact faces (versus scrambled faces) were presented. a. Marmoset 1 cm b. Macaque c. Human -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 15 --8 0 100 200 300 400 Time (ms) Diving Deeply Into Facial Perception Does the Visual System Contain Other Category-Specific Systems? **b** 1.0 0.8 0.6 0.4 0 0 10 20 30 40 Number of faces Testing Causality 100 50 0 a. FFA stimulation b. Color area stimulation rOFA rLO No TMS 3 2.5 2 1.5 1 0.5 rLO rEBA No TMS 3 2.5 2 1.5 1 0.5 0 0 c. Objects and bodies d. Faces and bodies TAKE-HOME MESSAGES - Neurons in various areas of the monkey brain show selectivity for face stimuli. - Similar specificity for faces is observed in fMRI studies in humans, including an area in the right fusiform gyrus of the temporal lobe: the fusiform face area (FFA). - Just as the FFA is specialized for processing faces, the parahippocampal place area (PPA) is specialized for processing information about spatial relations or for classifying objects based on spatial properties (e.g., an indoor versus outdoor scene), and the extrastriate body area (EBA) and the fusiform body area (FBA) have been identified as more active when body parts are viewed. 5. ### Failures in Object Recognition Subtypes of Visual Agnosia 100 90 80 70 0  No visual agnosia Visual agnosia Patient group Organizational Theories of Category Specificity  b. **c d** Developmental Origins of Category Specificity \ TAKE-HOME MESSAGES - Patients with agnosia are unable to recognize common objects. This deficit is modality specific. Patients with visual agnosia can recognize an object when they touch, smell, taste, or hear it, but not when they see it. - Apperceptive visual agnosia is a failure in perception that results in the inability to recognize objects. - Integrative visual agnosia is a failure to recognize objects because of the inability to integrate parts of an object into a coherent whole. - Associative visual agnosia is the inability to access con- ceptual knowledge from visual input. - Visual agnosia can be category specific. Category- specific deficits are deficits of object recognition that are restricted to certain classes of objects. - There is a debate about how object knowledge is orga- nized in the brain; one theory suggests it is organized by features and motor properties, and the other suggests specific domains relevant to survival and reproduction. 6. ### Prosopagnosia Is a Failure to Recognize Faces 0.2 0.1 0 --0.1 a. Lateral response 0.2 0.1 0 --0.1 --0.2 2 1.5 1 0.5 0 Sighted: Right medial ventral region 0.2 0.1 0 --0.1 --0.2 2 1.5 1 0.5 0 Developmental Disorders With Face Recognition Deficits b. Medial response 1.5 1.0 0.5 0.0 1.5 1.0 0.5 0.0 0.8 0.4 0.0 --0.4 --0.8 0.8 0.6 0.4 0.2 0.8 0.7 0.6 0.5 0.0 --0.2 CP Ctrls  C. Control D. Autistic brains Processing Accounts of Prosopagnosia **b** V4 Faces a. In the study phase (top panel), participants learned the names that corresponded with a set of faces and houses. During the recognition test (bottom panel), participants were presented with a face, a house, or a single feature from the face or house. They were asked whether a particular feature belonged to an individual. b. When presented with the entire face, participants were much better at identifying the facial features. Recognition of the house features was the same in both conditions. TAKE-HOME MESSAGES - Prosopagnosia is an inability to recognize faces that can- not be attributed to deterioration in intellectual function. - Acquired prosopagnosia results from a neurological inci- dent such as stroke or inflammation. Congenital prosop- agnosia is a developmental disorder. CP individuals have had difficulty recognizing faces for their whole lives. - Holistic processing is a form of perceptual analysis that emphasizes the overall shape of an object. This mode of processing is especially important for face percep- tion; we recognize a face by the overall configuration - Analysis-by-parts processing is a form of perceptual analysis that emphasizes the component parts of an object. This mode of processing is important for reading, when we decompose the overall shape into its constitu- ent parts. #### Summary #### Key Terms #### Think About It 1. What are some of the differences between processing in the dorsal and ventral visual pathways? In what ways are these differences useful? In what ways is it mislead- ing to imply a functional dichotomy of two distinct visual pathways? 2. Ms. S. recently suffered a brain injury. She claims to have difficulty in "seeing" as a result of her injury. Her neurologist has made a preliminary diagnosis of agnosia, but nothing more specific is noted. To determine the nature of her perceptual problems, a 3. Review different hypotheses concerning why brain injury may produce the puzzling symptom of dispropor- tionate impairment in recognizing living things. What sorts of evidence would support one hypothesis over another? 4. As a member of a debate team, you are assigned the task of defending the hypothesis that the brain has evolved a specialized system for perceiving faces. 5. EEG is an appealing alternative to fMRI for mind read- ing because a patient does not have to be in a scanner for the system to work. Describe what kinds of prob- lems EEG might present for mind reading, and suggest possible solutions. #### Suggested Reading  **273**