Week 10: Mental Imagery PSYC 221 Fall 2024 PDF

Week 10: Mental Imagery PSYC 221 Fall 2024  Outline Basics of mental imagery The mental imagery debate: Definition and empirical evidence Mental imagery and memory Zoom-in on auditory mental imagery (the coolest kind of imagery…) ...

Week 10: Mental Imagery PSYC 221 Fall 2024  Outline Basics of mental imagery The mental imagery debate: Definition and empirical evidence Mental imagery and memory Zoom-in on auditory mental imagery (the coolest kind of imagery…)  Learning Outcomes Define mental imagery Describe how to assess people’s mental imagery abilities Identify and define types of mental imagery  What is mental imagery? Definition: The ability to imagine, in our mind, some perceptual/sensory experience, without a stimulus present Different senses Focus on visual, auditory mental imagery today Perceive external world Mental image (internal) How do you test for Visual Mental Imagery?  Testing for Visual Mental Imagery: Approach #1 Ask questions! Vividness of Visual Imagery Questionnaire (VVIQ) tests object imagery  Testing for Visual Mental Imagery: Approach #2 Make people do a Visual Mental Imagery task and record performance Paper folding task tests spatial imagery Mentally unfold paper: Where Fold paper and punch a hole should the holes be?  So…does everyone have Visual Mental Imagery? Most people have some mental imagery or varying intensity Aphantasia -> Phantasia -> Hyperphantasia No voluntary visual Vivid visual mental mental imagery imagery Congenital “Real-life” quality to Acquired visual images Imagery in dreams  Outline Basics of mental imagery The mental imagery debate: Definition and empirical evidence Mental imagery and memory Zoom-in on auditory mental imagery (the coolest kind of imagery…)  Learning Outcomes Explain what the focal point of debate is in mental imagery research Compare and contrast each side of the mental imagery debate Summarise ways to test the coding of mental imagery, including experiments that can be conducted (the gist!) Explain how evidence supports/challenges positions in the mental imagery debate  What is the mental imagery debate about? Core of the debate: How are mental images represented? What is the code? Two debated answers: Picture-based Linguistic-based vs code code Depictive code Propositional code “Descriptive code”  Depictive coding is all about pictures Depictive coding: ‘mental picture’ camp Imagine a visual scene, keeping original spatial relationships Analog code  Propositional coding is all about language Propositional coding: ‘abstract language’ camp Abstract propositions store relationships from visual scene Allows for depictive representations to emerge, but not fundamental LEFT (T, C) BEHIND (C, T)  What do people agree on regarding Mental Imagery? 1) People experience mental images! 2) People do use propositional representation in cognition 3) Mental imagery is not necessarily solely depictive  What do people disagree on regarding Mental Imagery? True form of Propositional representation (coding) Depictive Side Side How do you solve a disagreement/debate in science (psychology)?  Testing the debate: Mental Scanning What is scanning? Moving our vision/attention around a visual scene What is mental scanning? Moving our vision/attention around our mental image  Testing the debate: Mental Scanning Logic of Mental Scanning experiments: If mental images are coded as pictures (depictive), then response times on Supports mental scanning tasks should be similar to those of visual scanning tasks. Depictive Side Supports If mental images are coded as abstract language (propositional), then Propositional response times on mental scanning tasks should not be related to those of Side visual scanning tasks.  Testing the debate: Mental Scanning Experiment 1 End scan: Timer off Start scan: Timer on Record how long it takes to complete the scan Imagine scanning images Question: Do mental Memorise a bunch of scanning times map to images expected visual Kosslyn, 1973 scanning times?  Testing the debate: Mental Scanning Experiment 1 Question: Do mental scanning times map to 7 Visual Scan expected visual 6 scanning times? Time to scan (sec) Mental Scan 5 4 3 Far scan > Close scan 2 for BOTH visual and 1 mental scans 0 Close Scan Far Scan Distance of Scan Significance: Supports Depictive Code -> truly scan an image in our mind Kosslyn, 1973 Testing the debate: Mental Scanning Experiment 1 Challenge People might be scanning a list of propositions, not mental pictures Hang on a minute! Further items in a picture = further items on a list: Back wheel stored closer to Bumper than Windshield Propositionalist Longer mental scanning times are consistent with the Propositional Side, too! Testing the debate: Mental Scanning Experiment 1 Challenge So…what next? Conduct a study that tests spatial relationships in mental images without a bunch of in-between features Testing the debate: Mental Scanning Experiment 2 Task: Memorise the landmarks Scan between landmarks in mental image Measure: Time it takes to scan scene Question: Do mental scan times map to visual scan times?  Testing the debate: Mental Scanning Experiment 2 Question: Do mental scan times map to visual scan 5 Visual Scan times? 4 Time to scan (sec) Mental Scan 3 Yes! 2 Same pattern as Experiment 1: 1 Far scan > Close scan for BOTH 0 visual and mental scan Close Scan Far Scan Type of Scan Interpretation: Scan pictures in our mind -> evidence for Depictive coding  Testing the debate: Mental Rotation What is rotation in space? Turning an object! What is mental rotation? Turning an object in your mind!  Testing the debate: Mental Rotation Logic of Mental Rotation experiments: Supports If you use a depictive code, then the further you have to rotate an object, the Depictive Side longer it will take you to mentally rotate an object. If you use a propositional code, then there will be no relationship between Supports Propositional how far you have to rotate an object and how long it takes you to mentally Side rotate an object.  Testing the debate: Mental Rotation Experiment 1 2 3 4 See an image Which image(s) match the original image? Question: When an image has to be rotated further, does it take longer? From Collins & Kimura, 1997 Testing the debate: Mental Rotation Experiment Question: When an image has to be turned further, does it take longer? Answer? Significance: Supports Depictive Side -> suggests we really manipulate pictures in our mind  Testing the debate: Mental Scaling What is visual scaling? Objects change in size and detail, depending on how close we are Due to space in field of view What is mental scaling? Imagined objects change in size and detail, dependent on how close/far-away we imagine being Mimics field of view changes  Testing the debate: Mental Scaling Logic of Mental Scaling experiments: The further you imagine being from an object, the smaller and Supports Depictive Side less detailed it is. You need to ‘zoom in’ to get more detail, and this should affect response times. The further you imagine being from an object should not affect Supports Propositional object size and detail. No need to ‘zoom in’ to get more detail, so Side response times are unaffected.  Testing the debate: Mental Scaling Experiment Bigger Smaller animal animal Question: Does it take longer to Question: Does it answer a question have legs? when the animal is smaller? Create scaling: Use image to Imagine pairs of answer questions animals about the animal Kosslyn, 1975  Testing the debate: Mental Scaling Experiment Question: Does it take longer to answer a question when the animal is Reaction time (msec) smaller? Smaller image response time > larger image response time Small target Big target image image Significance? Kosslyn, 1975 Testing the debate: Mental Scaling Experiment Challenge You told people to zoom in on images…so of course they did! Hang on a minute! Demand characteristics: When a participant does something (they think) the experimenter wants them to do Skews the results Propositionalist Experimenter expectancy: Experimenter’s expectations influence outcome People do not have to zoom in on pictures – but they can.  Testing the debate: Mental Scaling Experiment Challenge Imagery < perceptual Perceptual < imagery Research Test different sets of Question: Do assistants participants: How researcher (RA) good is perceptual vs expectations imagery acuity? influence outcome? Intons-Peterson, 1983  Testing the debate: Mental Scaling Experiment Challenge Perception Question: Do researcher RA Expectation: expectations influence Imagery < Perceptual Imagery outcome in mental imagery Acuity experiments? They CAN! Perception What does this mean for RA Expectation: the Mental Scaling Perceptual < Imagery Imagery evidence? Small Large Intons-Peterson, 1983 Size of image Mental Imagery debate cont’d: Do perception and imagery interact? Logic of Interaction experiments: If perception and imagery interact, then they share similar Supports Depictive Side mechanisms and are similar things (pictures). If perception and imagery do not interact, then they do not share Supports Propositional similar mechanisms and are not similar things (propositions). Side  Do Perception and Imagery interfere with one another? Perceive Imagine Question: Do people detect the stimulus better in the No Interference Interference conditions? conditions (same modality) Answer: Yes! No Interference When you have to perceive and conditions (cross imagine something in the same modality) modality, you do worse on the task -> shared mechanisms?  Recapping the Mental Imagery debate evidence Mental Scanning Experiments Depictive side: Evidence that we scan pictures in our mind Propositional side: Could be scanning lists in our mind Mental Rotation Experiments Depictive side: Evidence we rotate pictures in our mind Mental Scaling Experiments Depictive side: Evidence that we ‘zoom in’ on pictures in our mind Propositional side: Sure, but do we have to? Demand characteristics/expectations! Perception-Imagery Interaction Experiments Depictive side: Perception and imagery can interfere with one another BREAK  Recapping the Mental Imagery debate evidence Which side has more support? Depictive? Hang on a Science is about FALSIFICATION Propositional? minute! Evidence FOR a theory does NOT guarantee the truth of that theory Propositionalist  The Falsification Objection to Depictive Code Falsification: In science, we try to show that things are NOT the case Do not prove that things ARE the case Theory: Conclusion: All All swans swans are white are white (and I have proof!) Observe: Only white swans  The Falsification Objection to Depictive Code Falsification: In science, we try to show that things are NOT the case Do not prove that things ARE the case Theory: Conclusion: All All swans swans are NOT white are white (and I have proof!) Observe: Mostly white swans, one black swan  The Falsification Objection to Depictive Code Falsification: In science, we try to show that things are NOT the case Do not prove that things ARE the case Propositionalist’s aim: Show that Depictive representation as fundamental is NOT possible  Challenge to Depictive evidence Task: Look at Original Picture Use mental imagery to say whether the individual shapes were in the original shape Depictive code prediction: All shapes should be equally easy Propositional code prediction: Paralellogram should be harder because it isn’t in the linguistic code Challenge to Depictive evidence What do the data say? Which prediction is supported by these data? Evidence against depictive = stronger than evidence for depictive…if you buy the evidence!  The Mental Imagery Debate Conclusion Still some debate about Depictive vs Propositional as the fundamental coding of mental imagery Keep researching! Does mental imagery affect other cognitive abilities?  Outline Basics of mental imagery The mental imagery debate: Definition and empirical evidence Mental imagery and memory Zoom-in on auditory mental imagery (the coolest kind of imagery…)  Learning Outcomes Explain how theory supports empirical findings in memory and mental imagery Explain how you test theoretical predictions Independently find relevant information in a source  Mental Imagery and Memory In-class Assignment: W11-MI On your own: See a list of words Write down as many words as you can remember AFTER you see the list  Mental Imagery and Memory In-class Assignment: W11-MI On your own: See a list of words Write down as many words as you can remember AFTER you see the list In groups of 3-4: Answer 5 questions in W11-MI (~ 20 minutes)  In-class assignment: Mental Imagery can help memory Dual-coding theory: Can code things in pictures and/or language Dual coding = better memory Concreteness effect: Memory is better for concrete words (like tree and book) than for abstract words (like 60 hope and quest) Average correct recall 50 Explanation: Dual coding theory says we create 40 a picture AND a linguistic label for concrete words, but only a single linguistic label for (%) 30 abstract words 20 10 0 Concrete words Abstract words So far, we’ve focused on VISUAL mental imagery But, another common (and more awesome) form of mental imagery is Auditory ‘Imagery’ La la la… La la la…  Outline Basics of mental imagery The mental imagery debate: Definition and empirical evidence Mental imagery and memory Zoom-in on auditory mental imagery (the coolest kind of imagery…)  Learning Outcomes Differentiate voluntary from involuntary auditory imagery Explain how auditory imagery is affected by different acoustic features and describe research (briefly!) that supports these effects  Types of Auditory Imagery Voluntary Auditory Imagery: Asked to imagine a song or a rhythm Hear the notes in your mind Involuntary Auditory Imagery: Songs/rhythms get stuck in your head Hear the notes in your mind -> hard to stop Earworms!  Testing Voluntary Auditory Imagery: Probe tone experiments Traditional probe tone technique: Play a sequence of tones -> pause -> play a single tone Participant: Was the tone ‘correct’ in the sequence? Yes/No Participant: Was the timing of the tone ‘correct’ in the sequence? Yes/No Imagery probe tone technique: Play a few tones -> imagine tones-> hear a single probe tone Participant: Was the tone ‘correct’ in the sequence? Yes/No Participant: Was the timing of the tone ‘correct’ in the sequence? Yes/No Janata & Paroo (2006) Janata & Paroo (2006) How good are people at imagining Pitch and Rhythm? Listen to full sequence Pitch Rhythm Imagine part of sequence Proportion ‘Yes’ responses Proportion ‘Yes’ responses Note too low Note too high Note too early Note too late Correct note Correct timing  Does anything help us imagine rhythm timing? One idea: Motor system! We move a lot to music, so maybe when we move AND imagine rhythms, we get better at imagining the timing of rhythms  Actual Tempo: ~ 76 BPM Condition 3: Condition 1: Hear Condition 2: Imagine Imagine + tap song song song Hear a Hear a metronome Imagine tempo of slower/faster song version of song Imagine tempo of song Tap finger along Adjust to correct with imagined tempo Adjust metronome to tempo correct tempo of song Measured: How much the tempo deviated from the actual song Jakubowski et al., 2016  Tapping with a rhythm helps auditory imagery Condition 1: Hear song Condition 2: Imagine song Condition 3: Imagine song + tap # Participants # Participants # Participants % Tempo Deviation % Tempo Deviation % Tempo Deviation ~ 8% deviation ~ 26% deviation ~ 18% deviation from actual tempo from actual tempo from actual tempo Tempo accuracy: Hear song > Imagery + tap > Imagine only Jakubowski et al., 2016 Testing Involuntary Auditory Imagery: The case of Earworms! Lots of people report earworms Usually for familiar music, but can be novel (musicians/composers) Question: Do earworms actually resemble real music?  Testing Involuntary Auditory Imagery: The case of Earworms! A few times per day: Report earworm Question: How accurate is tapping? Tap tempo of an earworm Recorded on accelerometer Jakubowski et al., 2016  Testing Involuntary Auditory Imagery: The case of Earworms! Question: How accurate is tapping? Tapped tempo (bpm) Each circle represents a song that was an earworm What is the pattern in the data? Is earworm tempo accurate? Actual tempo of song (bpm) Jakubowski et al., 2016  Auditory Imagery take-aways It exists! Can be voluntary or involuntary Better at ‘hearing’ pitch inside our heads than rhythm BUT, depends if we engage the motor system Evidence that earworms sound like real music Familiarity effects?  Mental Imagery take-aways It exists! Continuum: Aphantasia -> Hyperphantasia The debate: Are mental images coded as pictures (depictive) or abstract language (propositions)? Evidence for/against each side Mental imagery interacts with other aspects of cognition Memory Textbook has other examples (emotions) Some properties of music are better imagined than others Next class: Language! Module 11