Week 3 Perception: How Do We Know What Is In The World? PDF

Week 3 Perception: How do we know what is in the world?  Outline 1. Sensation vs perception 2. Physiological mechanisms: How do sensory systems work? 3. Major theories of perception 4. Principles of perception – visual system 5. Modelling visual perception...

Week 3 Perception: How do we know what is in the world?  Outline 1. Sensation vs perception 2. Physiological mechanisms: How do sensory systems work? 3. Major theories of perception 4. Principles of perception – visual system 5. Modelling visual perception with AI 2  Outline 1. Sensation vs perception 2. Physiological mechanisms: How do sensory systems work? 3. Major theories of perception 4. Principles of perception – visual system 5. Modelling visual perception with AI 3 Sensation: Taking in information Sensory receptors 4 Sensory receptors respond to specific types of stimulation Receptor Type Stimulus Type Found in Chemoreceptors Chemicals Nose Mouth Skin + internal organs Thermoreceptors Temperature Skin Internal organs Mechanoreceptors Physical force Skin Ear Arteries Photoreceptors Light Eye  Why do we have multiple senses? Smell vs Touch/Temperature Vision vs Audition  Is it possible that our senses could have been different? Why? Electroreception: Sensing of stimuli using electrical fields Fish Water > air for electrical conduction Perception: Making sense of information Process + Sensory receptors Interpret 8  Which world are perceiving? Exteroception Interoception 9  Outline 1. Sensation vs perception 2. Physiological mechanisms: How do sensory systems work? 3. Major theories of perception 4. Principles of perception – visual system 5. Modelling visual perception with AI 10 Key steps in sensory perception Transduction Periphery -> Brain Cortical Processing 11  Chemical Senses: Gustation Sensory receptors in taste buds on tongue Sweetness, sourness, bitterness, Saltiness, and savouriness Distributed! Competing theories: Labelled line coding vs pattern coding Specific receptors + neurons that are active vs the pattern of activity 12  Chemical Senses: Olfaction Sensory receptors in olfactory epithelium of nose ~ one trillion distinct odours ! Direct communication from olfactory epithelium to olfactory bulb in brain Olfactory bulb -> hippocampus, amygdala… 13  14 Mechanoreception: Touch Somatosensory Cortex Sensory receptors in skin Skin -> spinal cord -> somatosensory cortex https://www.sciencedirect.com/topics/medicine-and- dentistry/primary-somatosensory-cortex  15 Somatotopic: Map of the body, proportional to nerve endings + function  16 Light Sense: Vision Sensory receptors in eye Focus light Control light amount Contains photoreceptors  17 Photoreceptors sense light Type of Location Function Receptor Rods Periphery Lightness of retina levels Cones Fovea Colour vision  Gathering Visual Information Saccade: Quick movement of the eyes from one location to another varied duration: 50-100 msec little info taken in during movements (saccades) maximise information Fixation: Brief period when the eyes stop moving and the visual scene is processed vary in duration (~ last 250 msec or less) ~ 3-4 per second Saccades and fixations take in important information in a visual scene 1 3 2 4  20 Information compression happens early Photoreceptors at back of retina Ganglion cells at front of retina Bundle together to form optic nerve to brain More photoreceptors than ganglion cells = information compression Tong et al. (2020) Early processing of visual signal  21 Visual cortex (V1) receives sensory information Optic nerve sends sensory information to primary visual cortex (V1) o Early cortical processing o Not end of line!  22 Hierarchical processing in the visual system Receptive fields: Not all stimuli activate all neurons equally Low-level features detected/processed by early neurons in V1 Higher-level features processed by later neurons after V1 Recall Hubel & Wiesel’s cat research Neurons respond preferentially to simple features Hierarchical model V1 = simple features processed  23 Hierarchical processing in the visual system  24 Can we localise object perception in late visual areas? Agnosia: Deficit in perception of a specific type of (visual) stimulus Prosopagnosia: Deficit in face perception; normal visual perception otherwise Functional localisation for faces? Kanwisher & Yovel, 2006  Expertise > Faces? Right Fusiform Face Area Size of Brain Response Bird experts and car experts Images of objects, cars, birds, faces Scan brain Objects Cars Birds Faces Image Viewed 25 Gauthier et al. (2000) In case you’d like to test yourself for prosopagnosia… https://www.testmybrain.org/face-blindness/face-blindness.html 26  Functional localisation of object recognition Dorsal = where Ventral = what Challenge: Are the pathways functionally dissociable? 27 Pitcher & Ungerleider, 2020  Verbal line matching task Goodale & Milner (1991) Studied ‘what’ and ‘where’ abilities of a patient with ventral pathway damage (‘what’ pathway) Action ‘mailing’ task 28  Patient Controls Verbal line matching task Goodale & Milner (1991) Impaired perceptual ability Unimpaired action ability Action ‘mailing’ task 29  Control Patient Distance between thumb and finger Shape matching task Control Patient Action pick-up task Alternate theory: Action vs Perception pathways 30  Outline 1. Sensation vs perception 2. Physiological mechanisms: How do sensory systems work? 3. Major theories of perception 4. Principles of perception – visual system 5. Modelling visual perception with AI 31  In-class Assignment Download the in-class assignment from Blackboard (W03-PSYC221- fall2024-InClass.pdf) In a group of 3, provide answers to the questions You can write directly on the.pdf or make a separate word document – just make it clear which question you’re answering Names of all 3 group members in document, but DO NOT SUBMIT YET 32  Theory 1: Direct perception Gibson (1979) Perceiver needs to do very little work in interpreting the world Light hitting the retina contains highly organized information that requires little or no interpretation Incoming sensory information directly linked to action  Direct perception: Evidence Johansson (1973) Attached lights to the limbs & joints of an actor Then the actor performed a series of common actions in the dark Participants who viewed the movements were very accurate at stating what the action was! Conclusion: No inference required to perceive walking patterns -> only using immediate sensory information  Theory 2: Constructive Perception von Helmholtz (1867) – first to describe Construct mental models of the world ‘Best guess’ theory  Evidence for constructive perception: Biology Saccades Jumpy vs smooth percepts Different images from each retina Fragmented vs unified percepts  Evidence for constructive perception: Ambiguity in sensory stimuli “The observer sees the probability distribution of the possible sources of the visual stimulus” (Purves & Lotto, 2003, p. 22) Similar sensory receptor stimulation can be consistent with many stimuli Example: Sense two points touching your hand. Without looking, what is touching your hand? ? ?  Evidence for constructive perception: Illusions #1 Illusions: Incorrect inference (guess) made about the world Lightness Illusion: Which square is darker: A or B? Same amount of light hitting our retina BUT Apply knowledge about the light source Evidence for constructive perception: Illusions #1  Evidence for constructive perception: Illusions #1 Lightness Illusion  Evidence for constructive perception: Illusions #2 Speech-to-song illusion: What did you hear? What did you hear? https://deutsch.ucsd.edu/psychology/pages.php?i=212  Evidence for constructive perception: Illusions #2 Deutsch et al. (2011): Repetition 1 vs 10 Played sentence to participants 10x Repetitions Condition 1: All repetitions identical Identical Condition 2: 2nd-9th repetition change Song Rating pitch Speech Repetitions Question: Does the phrase go from Changed speech to song? Repetition 42  Evidence for constructive perception: Bistability Percepts fluctuate between one thing and another What do you see? 2D -> 3D, but more than one way to get there!  Evidence for constructive perception: Bistability Meaning-content reversal Perspective reversal 44 Different levels of processing contribute to perception Top-Down Processing (conceptually-driven, based on context) IDENTIFY PATTERN ANALYZE FEATURES Bottom-Up Processing VISUAL (data-driven) INPUT  Outline 1. Sensation vs perception 2. Physiological mechanisms: How do sensory systems work? 3. Major theories of perception 4. Principles of perception – visual system 5. Modelling visual perception with AI 46 What sorts of things does our brain want to know about the world? What do we need to perceive? 47  Gestalt principles of perception “The whole is greater than the sum of the parts” Rules that shape our (visual) perception Explain how we segment and group objects Top-down processes -> based in experience Combine with bottom-up processes 48  Image segmentation combines bottom-up and top-down processes Challenge: How do we divide a continuous New object? image into specific, discrete objects or regions? One solution: Edge detection Light-dark boundaries Early visual processing Bottom-up Limitation: Not all edges are a new object 49  Image segmentation combines bottom-up and top-down processes Challenge: How do we divide a continuous image into specific, discrete objects or regions? One solution: Gestalt Principle of Experience Example: Figure-ground assignment What is object and what is background? What cues might we use? 50  Image segmentation combines bottom-up and top-down processes Figure-ground assignment cues: 1) Convex = object, concave = background 2) Symmetrical = object, asymmetrical = background 3) Smaller = object, larger = background Experienced-based: Come from what we know about objects 51  A number of Gestalt rules influence visual grouping Challenge: How do we combine a bunch of pieces of a visual scene into a single visual percept, like an object? Solution: Visual grouping detect larger patterns from smaller parts of the visual scene Umbrellas, people, buildings, trees… Gestalt rules: Similarity, proximity, good continuation, common fate, closure, good contour 52  Rule of Similarity When things have shared properties, we group them together http://www.scholarpedia.org/article/Gestalt_principles#Good_gestalt_principle 53  Rule of Proximity When things are close in space, we group them together How does this rule relate to the very thing you’re doing right now? 54  Rule of Good Continuation When things are continuous, we see them as one thing 55  Rule of Common Fate When things are behaving similarly, we see them as one thing Concerns motion or direction 56  Rule of Closure When things are missing full closure, we ‘fill in’ the missing part We form a closed object 57  Multiple cues determine depth perception Stereopsis = depth perception 2-D Challenge: Reconstruct third dimension from two-dimensional image Discuss bottom-up cues: Occlusion Motion parallax Binocular disparity 3-D Hollow-face illusion 58  Occlusion as depth cue Closer things block further things Monocular cue 59  Motion parallax as depth cue An object’s rate of change is different depending on its distance from a perceiver Image on retina changes at different rates Closer object moves further across field of vision Further object moves less across field of vision Monocular cue 60  Binocular disparity as depth cue Each eye receives slightly different input from the world Angle at which light hits each retina is different Use angle cues as depth cues Compare distances: Binocular cue Purple – orange Purple - green 61  Top-Down depth cues Hollow-face illusion Expectations about what things look like 62  Theories of objection recognition Object recognition: Match a stimulus in Stimulus Stored Representation the world with a stored representation ? Challenge: How do we do this matching? Proposed theories Template theory Feature theory 63  Template theory makes exact comparisons Input Template Point-by-point comparison between what we sense in the environment and what is stored A A a a Incoming pattern treated as an unanalyzed set of features 64 Would we recognize the stimulus as a banana in the Template Theory? Stored Stimulus Representation ?  Template theory has major limitations Template must be same size & position A vs a Large variability in patterns AAAAA Templates don’t allow for a description of how patterns differ P vs R  Feature theory focuses on component parts Some features are invariant (don’t change) Use these features to recognise objects Congruent with how neurons respond in the visual system  Feature theory has limitations, too We can describe patterns that do not have features Gestalt: pattern is more than the sum of its parts  Problems for Feature Theories Variability in stimuli Same patterns may have more than one description. Contextual factors contribute to object recognition Barenholtz (2014): Varied contextual cues to see if it affected object recognition  Contextual factors contribute to object recognition Low resolution Context > No context Familiar context > Unfamiliar context High resolution No Unfamiliar Familiar Context Context Context 71  In-class Assignment In the same group of 3, revise any answers you think require revision, based on the lecture/textbook. Write your answers in a different colour font Submit one document to Blackboard with all names in the document 72  Outline 1. Sensation vs perception 2. Physiological mechanisms: How do sensory systems work? 3. Major theories of perception 4. Principles of perception – visual system 5. Modelling visual perception with AI 73 Convolutional Neural Networks: Modelling Object Recognition Based on neural responses to specific features in the environment Convolution = fancy multiplication Input: Visual Kernel: Does Output value: Image computations Decision 74 75 Image Kernel – like a filter 4 5 7 2 0 0 0 0 2 9 7 5 6 1 5 5 0 1 3 7 9 2 4 5 7 2 0 0 0 0 2 9 7 5 6 1 5 5 0 1 3 7 9 2 4 5 7 2 0 0 0 0 2 9 7 5 6 1 5 5 0 1 3 7 9 2 4 5 7 2 0 0 0 0 2 9 7 5 6 1 5 5 0 1 3 7 9 2 4 5 7 2 0 0 0 0 2 9 7 5 6 1 5 5 0 1 3 7 9 2 4 5 7 2 0 0 0 0 2 9 7 5 6 1 5 5 0 1 3 7 9 2 4 5 7 2 0 0 0 0 2 9 7 5 6 1 5 5 0 1 3 7 9 2 4 5 7 2 0 0 0 0 2 9 7 5 6 1 5 5 0 1 3 7 9 2 Horizontal 4 5 7 2 0 0 0 0 2 9 7 5 6 1 5 5 0 1 3 7 9 2 edge kernel 4 5 7 2 0 0 0 0 2 9 7 5 6 1 5 5 0 1 3 7 9 2 4 5 7 2 0 0 0 0 2 9 7 5 6 1 5 5 0 1 3 7 9 2 4 5 7 2 0 0 0 0 2 9 7 5 6 1 5 5 0 1 3 7 9 2 4 5 7 2 0 0 0 0 2 9 7 5 6 1 5 5 0 1 3 7 9 2 4 5 7 2 0 0 0 0 2 9 7 5 6 1 5 5 0 1 3 7 9 2 4 5 7 2 0 0 0 0 2 9 7 5 6 1 5 5 0 1 3 7 9 2 4 5 7 2 0 0 0 0 2 9 7 5 6 1 5 5 0 1 3 7 9 2 76 Image Feature Map: Presence/absence of a visual feature in space 500 64 54 30 23 58 46 23 20 0 46 76 35 64 94 0 8 9 0 9 1 4 8 2 75 84 02 9 92 82 84 85 18 10 0 35 62 64 1 1 37 7 6 4 7 8 1 55 8 6 67 65 34 5 9 6 5 6 47 6 2 14 8 5 649 23 54 56 34 15 15 0 21 50 55 61 2 0 46 51 4 8 8 4 7 65 2 35 4 21 56 4 55 21 41 34 1 51 51 68 3 8 1 6 7 16 13 78 1 94 1 50 0 1 56 15 06 66 15 8 5 9 6 15 66 7 137 76 74 78 4 54 6 3 21 4 57 6 47 6 47 156 6 6 5 45 1 8 6 44 2 6 54 52 63 5 71 15 9 7 41 0 3 4 8 4 4 23 54 56 34 15 64 23 54 56 34 15 64 23 54 564 4 8 9 4 8 9 5 74 8 46 64 5 45 3 64 56 55 5 52 16 3 877 16 6 649 23 54 56 34 15 15 0 64 23 54 56 34 15 64 23 4 8 8 9 4 8 9 5 45 3 64 56 55 5 52 16 3 87 31 51 0 16 50 6 7 7 30 649 23 54 56 34 15 64 23 54 56 34 15 64 23 54 564 4 8 9 4 8 9 5 45 3 64 56 55 5 52 16 3 87 31 51 63 15 0 6 7 7 30 5 52 16 3 87 31 51 63 15 0 16 50 0 23 54 564 7 7 30 156 06 66 15 8 5 4 26 93 47 7 41 16 16 23 54 15 66 8 2 6 7 77 Combine kernel computations Recognise input image 8 78 Which aspect(s) of the human sensory/perceptual system are we modelling with these convolutional neural networks? 79  Next week: Auditory Perception & Music Cognition **Readings on Blackboard** 80