PsyBSc4 Allgemeine Psychologie I Perception I Lecture Notes PDF

PsyBSc4: Allgemeine Psychologie I Perception I Session 3: 31.10.2023 Dr. Sandro Wiesmann Some vocabulary excitatory = exzitatorisch (erregend) inhibitory = inhibitorisch (hemmend) converge = zusammenlaufen preferred = bevorzugt edge = Kante stripe = Streifen elongated = länglich intersection = Kreuz...

PsyBSc4: Allgemeine Psychologie I Perception I Session 3: 31.10.2023 Dr. Sandro Wiesmann Some vocabulary excitatory = exzitatorisch (erregend) inhibitory = inhibitorisch (hemmend) converge = zusammenlaufen preferred = bevorzugt edge = Kante stripe = Streifen elongated = länglich intersection = Kreuzung heuristic = Heuristik ("rule of thumb") interplay = Zusammenspiel environment = Umwelt attribute = zuordnen/zuschreiben recognize = erkennen experience = Erfahrung expectation = Erwartung feature = Eigenschaft infer/inference = schlussfolgern/Schlussfolgerung guess = vermuten, schätzen, raten likelihood = Wahrscheinlichkeit pattern = Muster enclosure = Umgrenzung proximity = Nähe occlusion = Verdeckung familiar = bekannt, vertraut 2 Schedule Session Date Topic Goldstein Chapter 1 17.10.2023 Introduction to Cognitive Psychology 1 2 24.10.2023 Neural Basis of Perception 2 3 31.10.2023 Perception I 3 4 07.11.2023 Perception II 3 5 14.11.2023 Perception III 3 6 21.11.2023 Attention I 4 7 28.11.2023 Attention II 4 8 05.12.2023 Memory 5 9 12.12.2023 Knowledge 9 10 19.12.2023 Visual Imagery 10 11 09.01.2024 Language 11 12 16.01.2024 Problem Solving 12 13 23.01.2024 Reasoning and Decision Making 13 14 30.01.2024 TBA 15 06.02.2024 Summary, exam preparation, questions 3 Today’s menu Part 1: Recap of the neural basis of perception Part 2: Mid-level vision  Bottom-up and top-down contributions to perception  Gestalt principles  Physical and semantic regularities governing perception In part based on Wolfe et al. (2021) Sensation & Perception, chapters 4 & 6.1 4 Part 1: Recap Five (rough) steps of visual processing 1. Light is transduced into neural signals in photoreceptors of the retina  Rods vs. cones, fovea vs. periphery, scotopic vs. photopic vision 2. Ganglion cells summarize information from several photoreceptors  Doughnut-shaped receptive fields, centre vs. surround, on-centre vs. off-centre ganglion cells 3. Ganglion cells send information to LGN cells  Same doughnut receptive fields as ganglion cells, but feedback from higher-level areas 4. Striate cortex cells summarize information from LGN cells  Building “line detectors” from doughnuts, orientation tuning, simple vs. complex cells, end-stopped cells 5. Higher-level brain areas (and their cells) responding to increasingly complex and specific stimuli  Faces, places, body parts, hands, etc. 6 From photoreceptors to doughnuts  Ganglion cells summarize signals from several photoreceptors  Ganglion cells have a “doughnut” shaped receptive field consisting of a centre (inner circle) and surround (ring) To ganglion cell 7 Ganglion receptive fields differ across the retina This peripheral ganglion cell has a large receptive field This foveal ganglion cell has a small receptive field Meist Human retina: Ca. 150 million photoreceptors, up to 1.5 million ganglion cells 8 Ganglion cell receptive fields (“doughnuts”)  Ganglion cell receptive fields (doughnuts) consist of ON (excitatory) and OFF (inhibitory) areas:  Shining light on the ON area of the receptive field increases the firing rate of the cell (excitation)  Shining light on the OFF area of the receptive field decreases the firing rate of the cell (inhibition)  ON-centre cells respond strongest to a dot of light surrounded by darkness  OFF-centre cells respond strongest to a dark dot surrounded by light ON-centre cell OFF-centre cell – + – + – – + – + + LGN cells have the same doughnut-shaped receptive fields! 9 From doughnuts to lines Striate cortex cells summarize signals from LGN cells whose receptive fields form a straight line LGN cell 1 LGN cell 2 LGN cell 3 LGN cell 4 Striate cortex cell (activation = sum of LGN cell signals) – – + – – – – + – – – – + – – – – + – – 10 From doughnuts to lines Striate cortex cells summarize signals from LGN cells whose receptive fields form a straight line Receptive fields of LGN cells converging on a single striate cortex cell form a straight line of “doughnuts” in our visual field! 11 From doughnuts to lines Striate cortex cell is maximally activated if all LGN cells it receives input from are activated LGN cell 1 LGN cell 2 LGN cell 3 LGN cell 4 – Striate cortex cell – + – (activation = sum of – – LGN cell signals) – + – – – – + – – – – + – – Light falls on ON centre of all LGN cells  LGN cells fire rapidly  Striate cortex cell fires rapidly 12 From doughnuts to lines Striate cortex cell is activated less if not all LGN cells it receives input from are activated LGN cell 1 LGN cell 2 LGN cell 3 LGN cell 4 – Striate cortex cell – + – (activation = sum of – – LGN cell signals) – + – – – – + – – – – + – – If stimulus is rotated, light falls on only some of the LGN cells’ receptive fields  Not all LGN cells fire rapidly  Firing rate of striate cortex cell is reduced 13 Orientation tuning of striate cortex cells Striate cortex cells are “tuned” to specific orientations of lines (they respond maximally to that orientation) Cell responds strongest to its preferred orientation Cell does not respond to nonpreferred orientations 14 Receptive fields of striate cortex cells are elongated – – + – – – – + – – – – + – – – – + – – – – – – + + + + – – – – Since activity of the striate cortex cell is the “sum” of LGN cell activity, we can imagine the receptive field of the striate cortex cell as the “sum” of the doughnuts 15 Receptive fields of striate cortex cells are elongated – – – – + + + + – – – – Remember: Each cell (ganglion, LGN, striate cortex) has a receptive field that corresponds to a part of the retina and therefore the visual field we look at! 16 There are different types of striate cortex cells Simple cells  Clearly defined excitatory and inhibitory receptive field regions Complex cells  No clearly defined excitatory and inhibitory receptive field regions  Will only fire if light falls on excitatory region  Will fire if light falls anywhere within the receptive field 17 There are different types of striate cortex cells Simple cell excitatory area (in red) Simple cell fires less because stimulus is outside excitatory area 18 There are different types of striate cortex cells Complex cell fires the same regardless of where the stimulus is within the receptive field because it does not have a clearly defined excitatory area 19 There are different types of simple cells Edge detectors  Respond strongest to edges (light on one side and darkness on other side of receptive field)  This edge detector will not respond to light on the right and dark on the left! Stripe detectors  Respond strongest to stripes of light with a certain width surrounded by darkness  This stripe detector will not respond to stripe of dark between patches of light! Receptive field + + + – – – – – – + + + Stimulus that would activate this cell 20 – – – There are different types of simple cells Edge detectors Respond to light–dark or dark–light Stripe detectors Respond to light–dark–light or dark–light–dark 21 Striate cortex cells can be end-stopped  End-stopped cells prefer bars of light of a certain length  When length is increased beyond size of the receptive field, response rate is decreased 22 Overview of different cell types Location Name Receptive field Orientation Responds best to tuning Retina Ganglion cell Doughnut shape No LGN LGN cell Doughnut shape No Simple cell Elongated, clearly divided into excitatory/inhibitory Yes Stimulus exactly on ON/OFF region  Edge detector One excitatory, one inhibitory area forming a clear edge Yes Edge between light and dark  Stripe detector Excitatory stripe between two inhibitory areas (or vice versa) Yes Stripe of light between dark (or vice versa) Complex cell Elongated, not clearly divided into excitatory/inhibitory Yes Stimulus anywhere within receptive field End-stopped cell* See simple and complex cells Yes Stimulus of specific length Striate cortex Dot of light surrounded by darkness (or vice versa) *Both simple and complex cells can be end-stopped or not! 23 Questions? How we can study the brain behaviourally When do ON-centre ganglion cells fire? ON-centre cell –  Generally, ON-centre ganglion cells are firing more…  The more light is on the centre (ON)  The less light is on the surround (OFF), it is in the dark  Strongest response: “A small dot of light surrounded by darkness” – + – –  Lower response when there is more light on surround! More light on surround  Cell fires less rapidly 26 Hermann Grid To explain the illusion, we must explain: 1. Black dots appearing at intersections in periphery 2. No black dots appearing between intersections 3. No black dots appearing when fixating intersections Can we explain all this with ganglion cells? 27 1. Black dots appear at intersections in periphery  In the periphery, ganglion cells have large receptive fields + - - -  At intersection, large parts of the OFF surround are stimulated with light (from the white grid lines)  Stimulation of OFF surround decreases the cell’s firing rate, we perceive the intersection as darker 28 2. No black dots between intersections in periphery  In the periphery, ganglion cells have large receptive fields + - - -  Between intersections, smaller parts of the OFF surround are stimulated with light (white grid lines)  The cell’s firing rate is not decreased as much, we perceive the grid lines between intersection as white 29 3. No black dots when directly fixating intersection  At the fovea, ganglion cells have small receptive fields that are completely in the light  Therefore, the response of these ganglion cells does not differ when we look directly at or between the intersections  We perceive any part of the grid lines as white when we directly look at it! 30 The ganglion explanation has a problem…  Illusory “black dots” disappear when using wavy lines! 31 The ganglion explanation has a problem…  Ganglion cell explanation cannot account for this! - - - + - + - -  Illusion seems to depend on straight lines: Probably related to cells in striate cortex! -  Curvy lines do not change the activation of centre vs. surround - We can “localize” visual phenomena in the brain without using neuroimaging or animal studies! 32 Questions? Part 2: Mid-level vision  Bottom-up and top-down contributions to perception  Gestalt principles  Physical and semantic regularities governing perception Low-, mid- and high-level vision  Low-level vision: Extracting basic image features such as edges, lines, patterns, etc. from the image (simple yet abstract stimulus properties)  Striate cortex (primary visual cortex or V1)  Mid-level vision: Grouping basic low-level features together into objects, identifying boundaries and surfaces in an image  A lot of brain areas you don’t need to know in detail…  High-level vision: Identifying complex stimuli such as faces, places, body parts  Inferior temporal cortex (inferior = below/lower): FFA, PPA, EBA, etc. 35  Low-, mid- and high-level vision 36 The challenge of mid-level vision Mid-level vision groups receptive fields of cells that respond to individual objects together. 37 Key function of mid-level vision: Boundary detection  Dark-light edge inside receptive field (red oval) is the same in all three images  A V1 cell would respond similarly to all three images  A V2 cell with the same receptive field would respond differently to (B) and (C)!  V2 performs figure-ground assignment (black object, grey background or vice versa) 38 Human boundary detection is powerful! We clearly see an arrow with a solid boundary! A computer has difficulties detecting this boundary… 39 Human boundary detection is powerful! We clearly see an arrow – even though the arrow has the same white colour as the background! 40 How do we know what is figure and background?  Surroundedness: Background usually surrounds figure  Size: Figure is usually smaller than background  Symmetry: Figure is more likely to be symmetrical  Parallelism: Parallel contours more likely to be figure 41 Today we’ll go from detecting lines and edges to recognizing objects… Part 2: Mid-level vision  Bottom-up and top-down contributions to perception  Gestalt principles  Physical and semantic regularities governing perception What do you see? Why do we see (a) as (b) and not as (c)? 44 Bottom-up vs. top-down processing  Perception is always an interplay of…  Bottom-up processing: Information about the environment as represented by the senses  Top-down processing: Information an individual brings Top-Down Knowledge Expectation Experience Memories Culture  The weight attributed to both sources of information varies! Interpretation Sound Colour Features Location Movement Bottom-Up 45 Dynamic nature of perception Perception is a dynamic and interactive process! 46 An attempt to define perception… Perception is…  The process of recognizing, organizing, and interpreting information from senses  Not an exact copy of the world!  Based on our past experience and expectations We will (once again) focus on visual perception, but many of the things we’ll discuss apply similarly to other senses! 47 Approaches to understanding perception Direct perception theories  Focus on bottom-up processing  Perception comes from stimuli in the environment  Parts are identified and put together, then recognition occurs Constructive perception theories  Focus on top-down processing  People actively construct perceptions using information based on expectations 48 Direct perception theories Biederman (1987): Recognition-by-components theory (RBC theory)  Objects are recognized by perceiving elementary features (“geons”): simple 3D shapes  36 different geons identified  Compare to idea that neurons along the neural hierarchy process increasingly complex stimuli 49 Constructive perspective  Focus on top-down processing, including…  Basing one perception on another  Inferences based on context  Guessing from experience  Top-down processing is usually quick and automatic (see next slides)! 50 Basing one perception on another Ebbinghaus size illusion One perception (size of orange circle) is based on another (size of black circles)! 51 Context effects on perception Demonstration in class Context drives perception of the ambiguous letter H/A! 52 Context effects on perception Demonstration in class 53 Guessing based on experience We can use our experience to make sense of ambiguous stimuli! 54 Perception relies on both bottom-up and top-down information! Part 2: Mid-level vision  Bottom-up and top-down contributions to perception  Gestalt principles  Physical and semantic regularities governing perception Back to the squares… What determines that we see (a) as (b) and not as (c)? 57 Helmholtz: Theory of unconscious inference  Image on the retina is ambiguous: How does the perceptual system “decide”?  Unconscious inference: Most likely interpretation is favoured (likelihood principle) based on unconscious assumptions, inferences about the environment  Process is rapid, feels almost automatic (even though it is complex!) 58 Following up on Helmholtz: Gestalt psychology  Wundt’s structuralism (see session 1):  Experience as a combination of basic elements (sensations)  “Perceptions are the sum of atoms of sensation”  Gestalt psychology (Wertheimer, Köhler, Koffka):  “The perceptual whole is more than the sum of its sensory parts”  The mind groups patterns according to rules of perceptual organization (“Gestalt principles”) These stimuli are more than just some of their parts! 59 Gestalt principles Good continuation  Points connecting to form straight or smoothly curving lines are perceived as belonging together  Lines are seen in a way to follow the smoothest path We see 1–4 and 2–3, even though other interpretations are possible! 60 Gestalt principles Prägnanz  Principle of simplicity or good figure  Patterns are seen in the simplest way possible We perceive five rings, not this… 61 Gestalt principles Similarity  Similar things appear to be grouped together  Colour, size, shape, orientation, etc. determine similarity  Grouping also as result of enclosure or proximity Similarity Enclosure Proximity 62 Gestalt principles Occlusion  When an object is partially covered by another, we perceive it as continuing behind that object Once again, we don’t perceive this… 63 Gestalt principles Familiarity  Elements are more likely to form groups when groups are familiar or meaningful Elements (e.g., fruit) form a meaningful group (a face) 64 Principle of familiarity Procrastination advice: Try to find all 13 faces! 65 Gestalt principles are heuristics!  Gestalt principles are heuristics (rules of thumb):  Provide best-guess solution to a problem  Fast  Often correct, but not always!  Compare algorithms:  Procedures guaranteed to solve a problem  Slow  Definite result 407 + 13 = ? 66 Why are heuristics useful?  Heuristics are useful to quickly derive a useful interpretation of a stimulus  Unlikely configurations are usually ignored (even though they are possible)  Makes everyday perception efficient! “Accidental viewpoint” 67 Why are heuristics useful? Visual input on our retina is always ambiguous – we have to settle on one interpretation! 68  Fun sidenote: Neurotree Sandro =) 69 Part 2: Mid-level vision  Bottom-up and top-down contributions to perception  Gestalt principles  Physical and semantic regularities governing perception Considering regularities of the environment  Gestalt psychologists: Gestalt principles are “intrinsic laws” (built into the system)  Effect of perceptual principles should be stronger than effect of experience with the world  Only minor role of experience  Modern psychology: Experience with the environment as a central component to perception!  Physical regularities: Perceived distance, light-from-above heuristic  Semantic regularities: Thematic relation and co-occurrence of objects 71 Perceived distance: Occlusion We know that the houses are behind the cars because the cars (partly) cover the houses 72 Perceived distance: Linear perspective We know that parallel lines converge in the distance 73 Perceived distance: Haze or aerial perspective Objects thar are far away appear “hazy” or “fuzzy” 74 Effect of perceived distance on perceived size 75 Effect of perceived distance on perceived size Ponzo Illusion: Which pair of horizontal lines shows lines of the same length? 76 Effect of perceived distance on perceived size 77 Effect of perceived distance on perceived size 78 Effect of perceived distance on perceived size Perceived size of an object is the result of:  Bottom-up processing:  Size of the image on the retina (remember: visual angle, relative!)  Top-down processing:  Perceived distance of the object  Size of object relative to other objects in the environment 79 Light-from-above heuristic We assume that light comes from above and therefore interpret shadows as cues towards 3D shape 80 Semantic regularities: Object co-occurrence What is this object? What is this object? We rely on scene context and object co-occurrence to identify (ambiguous) objects! 81 Summarizing the task of mid-level vision…  Group elements together that belong together (e.g., using Gestalt principles of similarity, proximity, etc.)  Distinguish between elements that do not belong together (e.g., boundary detection, figure– ground assignment, etc.)  Infer state of the outside world based on visual input using top-down knowledge (e.g., lightfrom-above heuristic)  Avoid interpretations based on highly specific, accidental combinations of features  Resolve ambiguity in the retinal image to come to a single interpretation 82 Questions? Summary  Low-level vision detects lines, edges and patterns in visual input. This process involves different types of cells in striate cortex.  Mid-level vision groups abstract features detected by low-level vision into objects suitable for identification by higher-level vision.  Perception is always an interplay of bottom-up and top-down processing, and it is never an exact copy of the world!  Context, prior experience, expectations, knowledge, etc. strongly influence our perception.  Gestalt principles, physical regularities and semantic regularities are heuristics used for efficient interpretation of ambiguous visual input. 84 What I want you to take away from this As we discussed last week, we often believe that seeing an apple must be the logical consequence of looking at it, even though the processes that accomplish this feat are quite complex. In this session, we discussed some of the basic principles that guide our understanding of visual stimuli and our everyday experience of the visual world. Importantly, we showed that perception is never an exact copy of the world (i.e., only bottom-up information) but instead strongly influenced by top-down knowledge. Additionally, we demonstrated that many of the principles guiding our perception are heuristics and do not always hold – our perceptual system can easily fail! The resulting illusions must not be understood as “errors” though, but as opportunities to learn how the visual system really works. The use of illusions and “errors” to study the mind is a theme that will be recurring throughout this lecture! Sample exam question The difference between simple and complex cells in striate cortex is that… a. the receptive field of simple cells is clearly divided into excitatory and inhibitory regions. b. simple cells only respond to edges, complex cells to stripes of light. c. complex cells respond to higher-level stimuli such as faces or places. d. simple cells respond best to dots of light, complex cells to lines of different orientations. 86 More information  Goldstein chapter 3  Wolfe chapters 4 and 6.1  Gestalt principles in (user interface) design  Check out artworks by M. C. Escher to experience your mid-level vision in action… 87 An invitation: Provide anonymous feedback  Provide feedback on the lecture “before it is too late”  Let me know what you like and what I should improve  Takes less than 5 minutes  Completely anonymous! https://sgl.uni-frankfurt.de/p4f/PsyBSc4_Feedback.html 88 Any questions? Feel free to ask questions or give me feedback after the lecture! That’s it – see you next week! Please do not look at the following slides until after the session! Context effects on perception (slide 52) Context determines whether we perceive an H or an A! 92 Context effects on perception (slide 53) Context determines whether we perceive a B or the number 13! 93 Guessing based on experience (slide 54) This “illusion” demonstrates how strong top-down effects (our experience with the world) on perception are. Even though the person is talking into a shoe, we interpret the scene and the objects therein in a meaningful way! 94 Semantic regularities: Object co-occurrence (slide 81) What is this object? What is this object? We rely on scene context and object co-occurrence to identify (ambiguous) objects! 95