Visual Perception & Imagery PDF

8️⃣ Perception I:Visual Perception & Imagery The Turn to the Brain Early models of how we see didn't talk much about how the brain does it. They were more about top-down analysis. exclusive of neural implementation In the 1980s, when we got ways to see how the brain works, scientists started studyin...

8️⃣ Perception I:Visual Perception & Imagery The Turn to the Brain Early models of how we see didn't talk much about how the brain does it. They were more about top-down analysis. exclusive of neural implementation In the 1980s, when we got ways to see how the brain works, scientists started studying what the brain does when we do different things. Also, one of the first ideas about this came from Ungerleider and Mishkin in 1982. They had a theory about two different systems in the brain for seeing. Two pathways of visual processing This hypothesis formed from experiments on monkeys in which different parts of their brain were selectively removed Perception I:Visual Perception & Imagery 1 Holds that there are two pathways of visual processing Dorsal stream: a system of interconnected regions of the visual cortex involved in the perception of spatial location, beginning with the striate cortex and ending with the posterior parietal cortex (WHERE) Ventral stream: a system of interconnected regions of visual cortex involved in the perception of form, beginning with the striate cortex and ending with the inferior temporal cortex (WHAT) Marr’s tri-level hypothesis (David Marr’s hypothesis) In addition, this information processing’s theory emphasized the importance of understanding how cognitive processes are implemented in the brain -holds that mental or artificial information-processing events can be evaluated on three different levels Computational level: Highest, most abstract level− What does the problem entail, i.e., what output is system is trying to get? What is purpose or reason for the process? Ex: Aim in visual processing is object recognition Algorithmic level: Programming level − What information-processing steps are being used to solve the problem? − Need a formal procedure that specifies how the data is to be transformed, what the steps are, and what the order of steps is Implementational level: Lowest level− Where is the hardware that is being used? How can the representations and algorithms be realized physically? Ex: computer hardware or human brain and neurons Marr’s model of visual processing Marr's theory of how we see: it's a step-by-step process that's based on different levels of understanding Perception I:Visual Perception & Imagery 2 It combines knowledge from psychology, math, brain science, and studies of people with brain injuries The system needs to take a messy group of things we see and make sense of them. These understood images then help with more complicated brain functions, like recognizing objects Drew on psychology, mathematics, neuroscience and clinical study of brain-damaged patients First stage: Image projected onto the retina is analyzed in terms of the intensity of areas of light and dark Adjacent regions of sharp contrast (of light and dark) indicate the presence of edges and contours The edges and contours determine the basic features of the object, including line segments and circular shapes The result is a raw primal sketch of the image Second stage: Similar features in the basic sketch are grouped based on size and orientation These feature groups are processed again to create a picture of the object, including its surfaces and layout, known as a 2.5-D sketch The 2.5-D sketch shows how far each point in the view is from the observer – it's focused on the viewer However, we need to be able to recognize that an object is the same, even if its image on our retina changes because either the object or the viewer moves Perception I:Visual Perception & Imagery 3 Third stage: The image is changed into a 3-D sketch. This sketch connects the parts of the object using lines of symmetry and elongation. Line of symmetry: a line that splits an object into two identical halves Line of elongation: a line that shows where most of the shape's mass is The 3-D sketch is focused on the object – the parts of the object are shown in relation to each other and are connected by shared features and lines. ➜ This solution helps us recognize an object from different angles and under different conditions like changes in light. Remember the problem of telling apart a Samoyed from a white wolf? Symmetry axis: line that divides an object into mirror image halves Elongation axis: line defining direction along which main bulk or mass of a shape is distributed Perception I:Visual Perception & Imagery 4 💡 Marr’s own work on vision contains relatively little discussion of neural implementation However, subsequent research on the mammalian visual system indicates that information in the visual cortex is in fact processed hierarchically Information flows through a progression of different areas, each of which generates a representation of increasing complexity Hierarchical organization of visual processing Human visual system Hierarchical Organization of Human Visual System First, the eye's retina sends information through the optic nerve to the brain's superior colliculus and then to the lateral geniculate nucleus (LGN) within the thalamus. The LGN sends this information to area V1 or the primary visual cortex, which is connected to the striate cortex, a unique part of the brain. The actual information processing starts in Area V1. V1 is organized in a way that nearby parts of what we see are processed by nearby parts of V1. Neurons in V1 are sensitive to basic features of what we see, like the direction things are moving and their orientation. Perception I:Visual Perception & Imagery 5 Ex: There are simple cells that react only to lines of particular orientation. This area also filters information to highlight edges and shapes. Area V1 connects to area V2 V2 neurons process similar features as V1, but also more complicated ones like combinations of edges, shape, and depth Depth is mainly determined by retinal disparity: the idea that points on objects at different distances from the viewer will land on slightly different spots on the two retinas The closer the object, the bigger the disparity This is the foundation for stereopsis or seeing in 3D 💡 In general, the extrastriate cortex (region surrounding the striate cortex) processes additional features of visual information, such as movement, spatial frequency, retinal disparity, and color From V2, information follows the ventral (“what”) or dorsal (“where”) pathway For ventral pathway:− Information goes from V2 to V4, then to inferior temporal cortex (ITC) Perception I:Visual Perception & Imagery 6 ITC includes specialized areas for face recognition (fusiform face area) and identification of the human body and body parts (fusiform body area) Coding of color Trichromatic (three color) Theory: retina has three types of color receptors (cones), each especially sensitive to one of three colors: red, green, and blue Most color blind people are not truly color blind; they simply lack functioning red- or green-sensitive cones They see the world in shades of yellow and blue Both red and green look yellowish to them Visual acuity is normal 💡 About 8% of Americans are red-green colorblind Opponent-Process Theory: color-sensitive receptor cells respond in an opposing center-surround fashion to pairs of primary color They are excited by light in the center and inhibited by light in the surround They are also excited by light of a particular color but inhibited by light of the opposing color Ø Ex: Some neurons are turned “on” by red but turned “off” by green ➜ This results in color afterimages phenomenon So which theory is correct? Both. Color processing occurs in two stages: Retina’s red, green, and blue cones respond in varying degrees to different color stimuli Cones’ responses are then processed by opponent-process cells Blindsight Perception I:Visual Perception & Imagery 7 ✧ Blindsight: Rare neurological condition where people who are blind in one or both visual fields due to damage to their visual cortex can nonetheless “guess” significantly above chance... The identity or location of particular objects The particular emotions expressed by a face in a photo in front of them eg. Blindsight patient was able to meander around all the clutter in a hallway that he was told was empty eg. Participants were asked to “post” a card into a slot eg. guess the animal in the picture presented in the blind field Proposed explanation for blindsight: There is a second pathway of visual perception that Does not go through the visual cortex Instead simply makes a very short loop through the limbic system: from the superior colliculus (visual processing center in brainstem) directly to the emotional/instinctual centers of the brain ➜Proposed mechanism for “intuition How are mental representations stored in the brain? 💡 The majority of research supports the analog viewpoint, but some people on some tasks use a propositional code. Also, some people tend to be visualizers and others tend to be verbalizers, and research has found neural correlates of these different cognitive styles, but more on that in the next lecture... The mental imagery debate (how is information stored): In analog code (i.e., as a pictorial representation) OR As a propositional code (i.e., descriptive) Perception I:Visual Perception & Imagery 8 Experiments on mental imagery by Roger Shepard and Jacqueline Metzler in early 1970s spawned the imagery debate Suggested that some types of cognitive information processing involve forms of representation that are very different from how information is represented in, and manipulated by, a digital computer Imagery and rotation studies: Rotate each object on the left to see if it matches the object on the right: ➜ Amount of time it takes to rotate a mental image depends on the extent of the rotation One feature of digitally encoded information is that the length of time it takes to process a piece of information is typically a function only of the quantity of information (the number of bits that are required to encode it) The particular information that is encoded ought not to matter But what the mental rotation experiments show is that there are informationprocessing tasks that take varying amounts of time even though the quantity of information remains the same ➜ This suggests that mental rotation tasks tap into ways of encoding information that are very different from how information is encoded in a digital computer 💡 More specifically, the information may be encoded in pictorial form, similar to the way a map represents a geographical region (analog code), rather than as a description (propositional code) Stephen Kosslyn found a similar effect in a different type of study Participants were asked to focus on a point in a picture then asked to answer questions about other parts of the picture Ex: Focus on the tail of the plane 1. 1) Does the plane have a propeller? 2. 2) Is there a pilot in the cockpit? ➜ Participants took longer to answer Question #1 than #2 Perception I:Visual Perception & Imagery 9 Length of time it took to answer varied according to the distance of the parts from the original point of focus So which viewpoint is correct: analog or propositional? Evidence in support of analog code (pictorial representation) Imagery and size: Condition #1: Imagine a rabbit standing next to an elephant Condition #2: Imagine a rabbit standing next to a fly “Does the rabbit have two front paws?” 💡 ➜ People make faster judgments about the characteristics of large mental images than of small mental images; also, they take longer to travel a large mental distance, whether that’s visual or auditory Imagery and interference: Create a clear mental image of a friend’s face Keeping that image in mind, simultaneously let your eyes wander over the scene in front of you 💡 ➜ Visual imagery may interfere with visual perception, and motor imagery with motor images Imagery and neuroimaging research: The primary visual cortex is activated when people work on tasks that require visual imagery Visual imagery activates about 70-90% of the same brain regions that are activated during visual perception o Similar findings have been reported for auditory and motor imagery People with prosopagnosia cannot create a mental image of a face Perception I:Visual Perception & Imagery 10 Evidence in support of propositional code (descriptive representation): Imagery and parts of figures: Look at the figure below, and form a clear mental image of the figure: Without glancing back at the figure in the previous slide, consult your mental image. Does that mental image contain a parallelogram? 💡 ➜ People have difficulty identifying that a part belongs to a whole if they have not included the part in their original verbal description of the whole Imagery and ambiguous figures: Create a clear mental image of the figure below: Write down what the figure in the previous slide depicted. Then give a second, different interpretation of the figure you saw 💡 ➜ Some ambiguous figures are difficult to reinterpret in a mental image Similarities between Human Visual System & Neural Networks Information-processing in the visual cortex is hierarchically organized – as are neural networks In addition, some parts of visual system are retinotopically organized – as are convolutional layers in convolutional neural networks Perception I:Visual Perception & Imagery 11 Perception I:Visual Perception & Imagery 12