Chapter 1: Challenges of Studying Perception PDF

Chapter 1. CHALLENGES OF STUDYING PERCEPTION Why Study Perception? ​ To understand yourself ​ Practical reasons: ​ design prosthetic devices ​ repair damaged sense organs ​ build artificial realities ​ design sensory environments ​ make techno...

Chapter 1. CHALLENGES OF STUDYING PERCEPTION Why Study Perception? ​ To understand yourself ​ Practical reasons: ​ design prosthetic devices ​ repair damaged sense organs ​ build artificial realities ​ design sensory environments ​ make technology easier to use ​ Engineering ​ You can only do these helpful things well if you understand how perception works Studying Perception is Complicated ​ Much of our brain is devoted to the complex processing that underlies perception ​ When scientists try to program computers to analyze visual scenes and recognize objects, the complications become very evident METHODS FOR STUDYING PERCEPTION Involves Many Sciences ​ Physics ​ To understand the kinds of external energy we can sense ​ Anatomy & physiology ​ To understand the operation of the brain and nervous system ​ Psychology ​ To understand cognitive functions like attention, memory, etc. ​ Information science ​ To examine how signals are extracted from noise. ​ Computer science ​ To construct models and test hypotheses by building systems Three Main Methods of Study 1.​ Physiological ​ (“Hardware” level of analysis) 2.​ Cognitive ​ (“Software” level of analysis) 3.​ Psychophysical ​ (Connects objective measurements to reports of subjective experience) Physiological Method ​ Asks: How are properties of objects in environment represented by activity in nervous system? ​ It examines relationships between stimuli & physiological responses ​ Characterizes activity in brain Neurons  ​ Neurons = units of processing ​ relationship between nerve impulses and specific perceptions ​ Specialized receptor gets info from outside world (pressure on skin) ​ The second neuron does not have specialized receptors. It has dendrites that can receive any nerve impulse ​ Signal is transmitted down the receptor cell’s axon to the next neuron The Neuron ​ Neuron = our unit of processing. ​ Everything in this course is based on: ​ the activity (firings, nerve impulses, action potentials), ​ and the interconnections, of these units How one neuron communicates with the next neuron ​ Neurons don’t actually TOUCH each other! ​ Synapse = gap between 2 neurons ​ PRE-synaptic neuron (before the gap) communicates with the... ​ POST-synaptic neuron (after the gap)... ​...by sending drugs (neurotransmitters) Neuron Firing Rates ​ Not a 1-to-1 relationship between stimulus and action potential ​ Firing RATE, not strength (magnitude) of each action potential carries info ​ There’s a spontaneous activity level = “resting rate” or “base rate” of firing (without stimulation) ​ If a neuron’s inputs encourage activation, the firing rate goes UP, above its base rate ​ If they encourage inhibition, the firing rate goes DOWN, below its base rate ​ Refractory period ​ One neuron communicates with the next neuron (and may affect the activity of that next neuron) ​ THIS IS HOW YOUR BRAIN PERCEIVES AND THINKS Overall Issues ​ Localization of function ​ WHERE in brain particular info is processed ​ Sensory coding ​ HOW features of environment = represented ​ specific neurons code for certain features? ​ pattern of firing (distributed over many neurons) codes for a feature? Physiological Investigative Techniques ​ Lesion Studies: ​ If a certain part of brain is damaged... ​ and particular ability disappears... ​ then that part of the brain is normally involved in this particular ability Single-cell Recording Technique  ​ Insert a delicate probe that can detect electric activity ​ Present specific stimuli ​ Measure the neuron’s response ​ This gives precise info about both timing and location of activity, but only in a narrow brain region Neuroimaging Techniques ​ Specifies functions for broader areas of brain ​ Activity corresponds to different perceptual tasks ​ Functional Magnetic Resonance Imaging (fMRI) measures the changes in blood flow that support increased neural activity ​ Note: blood flow to neurons increases shortly after the moment when neurons get more active (like muscles) ​ fMRI tells us precisely where brain activity occurs across a broad area, but is somewhat imprecise about timing Electro-Encephalography (EEG) ​ Measures electrical activity through the scalp/skull ​ Very precise information about the timing of neural activation across a broad area ​ Less precise about location of activity in brain Cognitive Method ​ Neisser ​ Thinking: info-processing flowchart ​ Two directions of info processing: ​ Bottom-up: construct a perception by analyzing info falling on receptors ​ Starts with receptors, info comes from senses and your brain creates an experience by analyzing whatever was perceived ​ Top-down: starts with analysis of high-level info (e.g., knowledge, goals) ​ When knowledge or experience influences your perception Psychophysical Method ​ Q: How we use info from environment (stimulus) to create perceptions (experience)? ​ Identify exactly what aspect of stimulus underlies some perceptual experience LIMITATIONS OF OUR PERCEPTION Do We Perceive “Reality” Directly? ​ To survive, we must acquire useful info about the world. ​ Perception constructs a “reality” for the perceiver (an internal representation). ​ There must be a systematic relation between it and the external world, or else we would not survive. ​ But, we do NOT perceive “reality” directly Interpretation  ​ Necker cube: same image can look different to: ​ different people ​ same person (different times) ​ Red dot close to you? (on front vertical surface) ​ Red dot far away? (on back vertical surface) ​ Hermann grid: gray dots at intersections? -- look right at dot: it disappears... Perception can be different from physical stimulus ​ 2D shape on Left looks longer/skinnier than the shape on Right? ​ Can’t turn off 3D interpretation of image, even if try ​ Face perception: impressive for right side-up faces; bad for upside-down faces ​ These two faces look not too different in expression? ​ I’ll flip the two pictures (turn each 180 deg around, in place) ​ We actively interpret the world Don’t Panic! ​ Easy to get carried away by optical illusions ​ Don’t ​ Yes, they show that our visual system has quirks ​ Vision is as accurate as it needs to be – as it evolved to be – for our survival PSYCHOPHYSICS Introduction ​ Psychophysics: determining quantitative relationships between ​ physical stimulus [external] ​ perception [internal psychological] ​ BIG PICTURE: we can’t study perception if we can’t even say whether an observer perceives a stimulus or not Gustav Fechner epiphany (1850) ​ Quantitative relationship between ​ mind (mental sensation) & ​ body (material/physical stimulus) Psychometric Function ​ Experimenter presents stimuli of different intensities ​ TASK = detection (did you see a light, or not?) Absolute Threshold ​ Absolute threshold = Smallest amount of stimulus energy necessary for the observer to detect a stimulus ​ PROBLEM ​ Thresholds aren’t all-or-nothing Difference Threshold ​ Difference threshold ​ (or JND, just- noticeable difference) ​ Smallest difference between 2 stimuli a person can detect SIGNAL DETECTION THEORY ​ absolute threshold: what intensity of stimulus produces 50% “yes-I- detected-it” responses? ​ But...is your sensitivity to the stimulus the only thing that determines your response? Response Criterion ​ Criterion = amount of sensory info observer requires for saying “yes” ​ Low criterion: say “yes” even if don’t perceive much evidence for the tone ​ liberal responder ​ High criterion: less willing to say “yes” ​ conservative responder Response is affected by: ​ Observer’s sensitivity (to the signal), ​ AND observer’s response criterion Catch Trials ​ Catch trials = no Target ​ (method of constant stimuli has none) ​ Not trying to “trick” observer, make look dumb ​ Ricky detects more tones (T-present trials) than Julian ​ But Ricky can’t TELL THE DIFFERENCE between TONE (target-present trials) and NO-TONE (catch trials) better than Julian Detection task ROC Curves ​ [ROC=“Receiver Operating Characteristic”] ​ Observers’ performance on detection task: ​ X-axis: %FA ​ Y-axis: %hits Same ROC curve ​ If we change the bias (e.g., w/payoffs), then for ​ same observer with the ​ same sensitivity, and even the ​ SAME STIMULI, ​ the observer may behave differently. ​ But their data point will still fall somewhere on the same ROC curve as before, because that shows their sensitivity.  ​ Same ROC curve: same sensitivity Different ROC curves ​ If 2 people’s data points fall on different ROC curves, the 2 people have different sensitivities ​ Who = more sensitive (higher hits, lower FA)? MAGNITUDE ESTIMATION ​ Relationship between the intensity of a stimulus, and the perception of its intensity ​ If you double the physical intensity of a stimulus, ​ does the stimulus now seem doubly bright or doubly loud? Chapter 2. NEURONS & PERCEPTION Inhibition ​ “inhibit” = stop something from happening ​ Activity of one neuron can increase or decrease activation in another neuron ​ Signals can be transmitted “sideways” across retina ​ not just “forward” through retina (receptors --> optic nerve) ​ “lateral inhibition” = the capacity of an excited neuron to reduce the activity of its neighbors Why care about low-level neural circuitry? ​ We detect stimuli, but we also assess the appearance of objects ​ Simultaneous contrast: Perceiving these two squares on different backgrounds as being different, even though they are physically identical What does it mean to say that “information” is “analyzed”? ​ Information is analyzed, interpreted, transformed ​ so that the resulting signal is easier for perceptual system to understand ​ Synapses “process” electrical signals as they travel from receptors along to further into the brain. ​ It’s not like photocopying Neuron B ​ CIRCUIT with excitation, convergence and inhibition ​ Leads to neuron B which responds best to a bar of light of a specific length ​ Neuron B responds best to a medium-length bar of light. ​ Worse to shorter or longer bars of light ​ NEURAL PROCESSING, by the circuit leading to neuron B, transformed the info. ​ From input (photoreceptor activations) ​ Into different output (response of neuron B) ​ Neuron B extracted info (how much evidence it collected for the presence of a certain-length bar of light) RECEPTIVE FIELDS Introduction ​ Receptive field (RF): region of retina that, when stimulated, influences firing rate of a particular neuron ​ Could make neuron fire more (excitation) ​ Could make neuron fire less (inhibition) ​ Which neuron? ​ It can be at any level where visual input is processed ​ We’ll be looking at: ​ Ganglion cells entering the optic nerve ​ Primary visual cortex (V1) Measuring a Neuron’s Response ​ Simultaneously: ​ Present visual stimuli ​ Record response of a single neuron ​ Answer: what visual stimuli does that neuron respond to ​ Area A: No light presented in Area A affects that neuron’s activity ​ Area A is NOT in neuron’s RF ​ Area B: light causes an excitatory response. ​ Mark with “+”. ​ Area C: light causes an inhibitory response ​ decrease in firing rate. ​ Mark with “-” ​ Areas B and C are the RF of the neuron we are studying Center-Surround Receptive Fields ​ Center-Surround RFs are characteristic of: ​ Retinal ganglionic cells ​ The LGN (low-level visual processing) ​ Excitatory-center, inhibitory- surround: common ​ Inhibitory-center, excitatory- surround: also exist 1.​ What will happen? ​ HIGHER firing rate than baseline, by a lot ​ HIGHER firing rate than baseline, by a little ​ LOWER firing rate than baseline, by a lot ​ LOWER firing rate than baseline, by a little 2.​ Whole inhibitory surround of RF is illuminated. No light falling on excitatory portion of RF. What will happen? ​ HIGHER firing rate than baseline, by a lot ​ HIGHER firing rate than baseline, by a little ​ LOWER firing rate than baseline, by a lot ​ LOWER firing rate than baseline, by a little CENTRE SURROUND INHIBITION EXPLAINS AN ILLUSION Explaining the Hermann Grid  ​ Perceiving things that aren’t there ​ Hermann grid: gray dots at intersections? -- look right at dot: it disappears Why is there no gray spot at fixation? (when you look right at an intersection) ​ Smaller RFs at fixation (fovea) ​ A small RF can fit within a single gridline ​ So, inhibition at the RF’s surround isn’t different between: ​ at an intersection ​ along a gridline ​ RF is filled with uniform light; the two neurons will emit identical responses ​ (with a small enough grid, dark spots would appear at ALL intersections, included the fixated one) ​ Lateral inhibition can explain many illusions of appearance, but: ​ Benary Cross: both gray triangles receive same amount of lateral inhibition (neighboring white/dark regions) ​ But B looks lighter than A THE SENSORY CODE Specificity Coding ​ Representation of a specific stimulus by the firing of a neuron specialized to respond to just it Population Coding ​ Representation of specific stimuli by pattern of firing of many neurons ​ Each neuron fires to each face, but by different amounts Sparse Coding ​ A particular object is represented by a pattern of firing in a small number of neurons Chapter 3: INTRODUCTION TO VISION The Stimulus for Vision: LIGHT The electromagnetic spectrum ​ Light energy emanates from light sources ​...e.g., the sun, a lightbulb ​ Light is reflected off of objects in the environment, and some of it then makes it to our eyes ​ The way that this light bounces off of objects... ​...provides clues about the properties of these objects ​ The visual system interprets this pattern of incoming light... ​...to determine the properties of these objects ​..but...what we perceive visually is filtered through the properties of the visual system FOCUSING THE INCOMING LIGHT ON THE RETINA ​ The eye is the structure that collects the incoming light from the environment ​ An image = made up of MANY points (~little pixels)  ​ To explain focusing the image, we explain in terms of what happens to just ONE of these points. ​ ACCOMMODATION: Lens changes shape so that images of nearby objects are not blurry on the retina ​ The lens of your eye is usually relatively flat (relaxed state) ​ That works well for focusing images of far-away things on the back of your eye ​ But when needed, muscles can make it rounder and fatter ​ This rounder shape curves light rays more than the flat shape does ​ And that’s exactly what you need so that close-up objects won’t cast blurry images on the back of your eye Accommodation ​ Ciliary muscles change the shape of the lens ​ Fatter lens bends light rays more sharply on way to retina First task: focus pattern of light onto retina at back of the eye. ​ Light is focused from one point in the world onto one point at back of eye (“focus point”) ​ And this happens for all points in the image ​ So the incoming image activates the retina’s receptors ​ And the image is focused rather than blurry, because of adjustment of the shape of the lens THE RETINA Receptors ​ Abt 20x more rods than cones Transduction ​ The transformation of one form of energy to another. ​ Nervous system converts ​ patterns of physical energy (light) ​ into neural events (photoreceptor electrical signal) ​ Part of the light-sensitive photopigment (in outer segment of photoreceptor) absorbs light and changes shape ​ --> causes receptor to generate electrical signal ROD & CONE ​ photoreceptors transform light energy into neural activity ​ the RODS are much more sensitive than the cones ​ But there are NO RODS at the center of your vision ​ RODS more sensitive than cones ​ But NO RODS at center of vision ​ ganglion are last layer and send out through optic nerve ​ Optic nerve leaves eyeball and transmits to rest of the brain ​ One blind spot on each eye (off to side) DARK ADAPTATION Major Theme ​ What the receptors do encode, vs. don’t encode... ​...affects what info can get in, for neural processing... ​...and thus affects what you can and can’t perceive. ​ Example: ​ Dark Adaptation = increase in sensitivity that occurs when illumination changes from light to darkness ​...What does dark adaptation DO to your visual receptors that helps you see better after time in the dark? 2 Systems ​ DARK ADAPTATION occurs in 2 stages: ​ faster stage due to adaptation of cones ​ slower stage due to adaptation of rods ​ …2 different systems controlling vision ​ coping with huge changes in light levels The Dark Adaptation Curve ​ Look off to the side FULL dark adaptation curve Dark adaptation curve: cones only ​ Look off to the side Dark adaptation curve for rods only ​ Threshold = smallest amount Rod-cone break ​ When lights go off, sensitivity of both rods and cones begins increasing... ​ At first, cones control vision, ~ 4 min to max sensitivity ​ Rods ~ 30 min to max sensitivity! ​ After ~7 min, rods control vision (more sensitive) Photoreceptor Pigments & Dark Adaptation ​ What is dark adaptation? ​ WHY when the light goes off does your visual sensitivity gradually increase? What happens during transduction? ​ Pigment absorbs a photon of light ​ The retinal detaches from the opsin ​ the whole thing changes (loses) colour  ​ ed -> orange -> yellow -> transparent / white ​ “PIGMENT BLEACHING” ​ Nerve impulse ​ Pigment must recover - retinal has to re-attach to opsin ​ “PIGMENT REGENERATION” ​ Must be near pigment epithelium ​ Pigments in receptors contain large opsin molecule, attached to retinal ​ Visual pigments must recover – Pigment Regeneration (reason why vision improves with time in the dark) ​ Rods adapt more slowly than cones because the rod pigment regenerates slowly ​ Visual pigment regeneration is responsible for increased sensitivity that occurs during dark adaptation (it’s why dark adaptation helps) Dark Adaptation: Rods vs. Cones ​ Move from bright light conditions (e.g., bright sun) into dim light conditions (e.g., indoor dark): ​ First you see almost nothing, then by ​ 30min later, you see much more ​ How can your retina detect light in such a HUGE range of light intensities? ​ It contains TWO systems ​ 2 kinds of light-sensitive photoreceptors Two Kinds of Photoreceptors ​ CONES evolved for daytime vision (high light levels) ​ Great for detecting light when there’s plenty of illumination ​ Pretty useless for detecting light in low illumination levels ​ Not very sensitive ​ RODS are for night vision (lower light levels) ​ Great for detecting light in low levels of illumination ​ Very sensitive - a rod can respond to a single photon of visible light ​ Pretty useless for vision in bright lighting ​ overloaded in bright light because they are so sensitive ​ CONES adapt faster than rods, ​ so for the first few minutes in the dark your visual experience is provided by your cones (“cone mediated vision”) that have adapted (some) ​ [your rods are still overwhelmed from having been in bright light before, and they haven’t regenerated much yet] ​ RODS adapt more slowly than cones, ​ but after a couple of minutes in the dark they are already more sensitive than the cones ​ So your visual experience is provided by your rods (“rod-mediated vision”) That major theme again... ​ You CAN’T see an object... ​...if it doesn’t emanate or reflect light that ends up activating rod and/or cone receptors in your retina ​ Info can only get in to brain if activates a receptor  ​ Properties of rod & cone photopigments play important role in shaping perceptions SPECTRAL SENSITIVITY Spectral Sensitivity Curve ​ … switch from showing ​ white light (contains all wavelengths of spectrum) ​ to monochromatic light (contains just one wavelength) ​ PERSON’s sensitivity to light at each wavelength ​ DARK ADAPTATION shifts vision ​ from cones ​ to rods ​ This affects spectral sensitivity ​ RODs are more sensitive to shorter- wavelength light ​ CONES are more sensitive to longer-wavelength light Absorption Spectrum ​ amount of light absorbed by a photopigment [SUBSTANCE], at each wavelength ​ rod spectral sensitivity curve (of PERSON) ​ absorption spectrum of the rod pigment ​ Spectral sensitivity of rod vision is due to absorption of light by rod visual pigment CONVERGENCE ​ more than one neuron sends signals to another neuron Rods converge more than cones so rods are more sensitive ​ Ganglion cell receives input from (average) 120 rods, vs. (average) 6 cones ​ At low light levels (high sensitivity needed): ​ a ganglion cell receiving input from many rods might still get activated (but not a ganglion cell receiving input from only a few cones) Cones converge less than rods so cones have better acuity ​ Acuity = ability to see details ​ All-cone fovea has good acuity ​ Visual acuity drops as move from fovea to periphery ​ Visual acuity also drops during dark adaptation (sensitivity increase): cone vision --> rod vision ​ Lots of CONVERGENCE gives rods low acuity & high sensitivity Practice Quiz: 1.​ For a psychometric function, the x-axis is ___ and the y-axis is ___ ​ Intensity of stimulus, percent detection 2.​ Which does pigment regeneration happen faster for ​ Cones 3.​ Which are relatively more sensitive to shorter wavelength light? ​ Rods 4.​ One particular neuron died and now I can’t recognize Kim Kardasian’s face, this would be evidence to support? ​ Specificity coding 5.​ Lots of convergence leads to high ___ but low ___ ​ sensitivity, acuity 6.​ What is the y-axis in an ROC curve? ​ Percent hits 7.​ You can find come receptors in ___; you can find rod receptors in ___ ​ Both the fovea and peripheral retina; the peripheral retina only 8.​ As stimulus intensity is increased, recording from a single neuron shows ​ The rate of firing of the nerve fibre increases