Syllabus Cognitive Psychology Chapters 1-9 PDF

Syllabus Cognitive Psychology (Chapters 1-9) For the course Biological and Cognitive Psychology 2023-2024 Dr. Richard Godijn 1 Table of Contents Chapter 1: Intro Biological and Cognitive Psychology………………………..4 Historical foundations of cognitive psychology…………………………………………………4 Towards a cognitive psychology……………………………………………………………………….5 A short history of biological psychology……………………………………………………………8 The speed of information processing……………………………………………………………..11 Towards a cognitive neuroscience………………………………………………………………….13 Chapter 2: Approaches to Cognitive Neuroscience………………….……..14 Reaction time methods………………………………………………………………………………….14 Event related potentials…………………………………………………………………………………18 Brain imaging methods…………………………………………………………………………………..22 Transcranial magnetic stimulation………………………………………………………………….25 Converging and conflicting evidence………………………………………………………………27 Chapter 3: From Neuron to Cognition…………………………………………….30 Measuring single cell activity………………………………………………………………………….30 Specificity coding versus distributed coding…………………………………………………...32 Neural networks…………………………………………………………………………………………….35 Predictive coding……………………………………………………………………………………………37 Chapter 4: From Brain to Cognition…………………………………………….….42 General overview of cerebral cortex functions……………………………………………….42 Prefrontal cortex…………………………………………………………………………………………….43 Working memory……………………………………………………………………………………………44 Working memory storage………………………………………………………………………….…..48 Interactions between prefrontal cortex and posterior regions……………………….52 Bottom-up and top-down processing in the brain………………………………………….54 Chapter 5: Cognition of Perception…….……………………………………….….56 Figure-ground and object organization…………………………………………………………..56 Gestalt laws of perceptual organization………………………………………………………….58 Likelihood……………………………………………………………………………………………………….63 Object constancy……………………………………………………………………………………………64 Size constancy and pictorial depth cues………………………………………………………….65 Brightness and color constancy…………………………………………………………………..….68 Multisensory integration………………………………………………………………………………..70 Predictive coding revisited……..…………………………………………………………………..….73 Chapter 6: Attention and Action……………………………………………………..74 2 Intro attention and action……………………………………………………………………………...75 Attentional limits …………………………………………………………………………………………..75 Hemi-spatial visual neglect …………………………………………………………………………….79 Actions aid perception …………………………………………………………………………………..82 Own motion versus object motion …………………………………………………………………83 The mirror neuron system ……………………………………………………………………………..84 Affordances ……………………………………………………………………………………………………87 Final remarks interactions attention perception and action …………………………..91 Chapter 7: Cognition of memory……………………………………………..……..92 Intro memory ………………………………………………………………………………………………..92 Context and cues …………………………………………………………………………………………..94 Familiarity and source monitoring …………………………………………………………………96 False memories ……………………………………………………………………………………………..98 Semantic networks ………………………………………………………………………………………102 Semantic memory in the brain …………………………………………………………………….104 Chapter 8: Memory processes…….…………………………………………..……106 Interference ………………………………………………………………………………………………..106 Protecting memories from interference……………………………………………………….110 Systems consolidation …………………………………………………………………………………112 Memory processes during sleep ………………………………………………………………….113 Reconsolidation …………………………………………………………………………………………..114 Emotional memories ……………………………………………………………………………………116 Chapter 9: Cognition of language …………………………………………..……121 Intro language…………………………………………………….………………………………………..121 Speech errors……………………………………………………..………………………………………..123 Phrase structure grammar……………………………….…………………………………………..124 Understanding words…………………………………………………………………………………..126 Understanding sentences……………………………………………………………………………..129 Positional noise for letters and words…………………………………………………………..133 Understanding texts and stories…………………………………………………………………..134 Concluding remarks……………………………………………………………………………………..136 References……………………………………………………………………………………137 3 Chapter 1: Intro biological and cognitive psychology What is happening in your mind as you read this sentence? If you are reading this on your laptop you are looking at your screen, you are looking at the words, perceptually processing the letters, the words, the sentences. You are moving your eyes from word to word, fixating many of them, but skipping others. You are attending to the words, retrieving their meaning from your memory. You are forming an understanding of the text and hopefully will remember at least some of what you read later on. You are using cognitive processes related to perception, attention, memory, action and language. Soon when you finish reading in this syllabus you will move on to other tasks, but whatever these tasks may be you will interact with your environment through cognitive processes. What is happening in your brain? For each of these cognitive processes there are brain regions involved. Whether you are reading this syllabus or doing something else, many brain regions will be activated. Neurons will deliver action potentials and will activate or inhibit other neurons. All cognitive activity is the result of activity in the brain. In other words, there is always a biological side to cognitive activity. This course is about how we interact with our environment. It is about cognitive psychology and biological psychology, the two fundamental aspects of our mental life and of our behavior. In this chapter I will introduce the fields of biological and cognitive psychology and present a historical overview of both fields all the way to their relatively recent integration in the field of cognitive neuroscience. Historical foundations of cognitive psychology Although cognitive psychology really only got going in the 1950s we must go back all the way to the beginning of experimental psychology in the late 19th century to truly appreciate the principles of the field. The basic questions of the different historical approaches have always been, what should we study and how should we do that? Psychology is the study of human behavior, which is controlled by the human mind. So how do we study the mind? This question has received different answers throughout the last century and a half. In 1879 the first experimental psychology lab was founded in Leipzig by Wilhelm Wundt. Thanks to his pioneering work he is often called the father of experimental psychology. Together with his student Edward Titchener he attempted to discover the structure of the mind. As the structure of matter was examined in physics and chemistry, it was thought that psychology should focus on examining the basic elements of the mind. Presumably ideas, perceptions and emotions have a structure, much like matter also has a structure. From a historical perspective Wundt and Titchener, and early experimental psychology in general, is often remembered for the method of introspection. In order to study the mind these psychologists thought it best to train subjects to look into their mind (to intro-spect) and to report what they observed. How else can we gain access to the perceptions, ideas and emotions of the mind? It was this method of introspection that came under heavy attack in the early 20th century, most notably by John B. Watson. According to Watson introspection cannot be the way to go for psychology for two important reasons. First, there is no way of verifying the results. If a subjects reports about what is going on in his or her mind, how do we know that this report is accurate? Second, the variability in such reports can be quite large. Do these differences reflect differences in the mind or do they reflect differences in the way the content of the mind is being reported? How can we know? In fact, Watson rejected the study of the internal mind and advocated the study of observable behavior instead. Watson’s approach became known as behaviorism. From his publication of The Behaviorist Manifesto in 1913 (Watson, 1913), all the way into the 1950s, the interest shifted from the content of consciousness to how human behavior is 4 shaped by experience, how we react to our environment and how our behavior changes as we learn about the relations between different stimuli, and later, how our behavior changes as we learn through reward and punishment. This meant an end to the study of consciousness, imagery, and mental states. Much of the behaviorist research focused on conditioning. Ivan Pavlov had already showed that a dog can be conditioned to produce saliva when a bell rings if the bell is followed by the experimenter bringing in food. So, the food (unconditioned stimulus) is normally followed by the production of saliva (unconditioned response), but the dog can learn that the bell (conditioned stimulus) is followed by the food, such that the bell becomes enough for the dog to produce saliva (conditioned response) in anticipation of the food. Similar learning processes occur in humans. A famous example was Watson’s demonstration that a child, little Albert, could be conditioned to fear a rat. Initially Albert was not afraid of the rat, but if a loud noise was presented just before the appearance of the rat and if this procedure was repeated several times, Albert started to be afraid of the rat. Albert did not like the loud noise (unconditioned stimulus) and showed a fear response (unconditioned response) to this noise. So Albert learned an association between the loud noise and the rat, such that the rat (conditioned stimulus) became enough for a fear response (conditioned response). It was Burrhus Skinner who took this conditioning one step further in the late 1930s (e.g. Skinner, 1938). He demonstrated the crucial role of reward and punishment in shaping behavior. In contrast to Watson’s classical conditioning, this operant conditioning showed that when behavior is rewarded we are more likely to repeat it, while behavior that is punished is less likely to be repeated. Decades of research in the early to mid 20th century showed how much of our behavior is the consequence of conditioning. The question can be asked, however, how far this approach to experimental psychology can get us. What about complex behavior like the production and understanding of language? Can this be explained through reinforcement? Skinner believed it could! In 1957 he published “Verbal Behavior” (Skinner, 1957), which laid out his behaviorist view of language acquisition. When a child learns language it imitates speech and learns that certain words or phrases produce certain outcomes. When the child learns its first words the parents will reinforce the speech with a smile and some words of encouragement. Later, speech is rewarded when the child learns to explain what they want. Even the attention given to the talking child could be seen as rewarding. The bottom-line in this behaviorist view is that language is learned through the way in which people respond to the verbal behavior. This approach shows just how far Skinner took behaviorism. For some, this went too far. In response to Skinner’s publication Noam Chomsky wrote a critical review (Chomsky, 1959) and argued that reinforcement played far less a role in language learning than Skinner had suggested. According to Chomsky humans have an innate ability to learn language, that they pick up on the syntactic structure of language, and that this does not rely on reinforcement. Chomsky noted universal characteristics of grammar, which, for him, point to some innate ability for organizing language (Chomsky, 1957). Furthermore, language is a creative process. Any speaker can create a sentence that has never been uttered before, and the audience can still understand the meaning of this sentence despite having never heard it before. Chomsky’s view of language learning suggested that the behaviorist view, useful as it might be, does have its limitations in the understanding of human behavior. Towards a cognitive psychology The study of language learning is only one field where the behaviorist shortcomings become apparent. It turns out that denying or at least ignoring mental processes only gets us so far. Take for example research into spatial navigation. How do we learn where we need to go? Do we just remember in which direction we need to move when we see where we are? In 1948 Edward Tolman 5 wrote a paper entitled “Cognitive maps in rats and men” (Tolman, 1948). Tolman studied the behavior of rats while they learnt their way around a maze. It became very clear to Tolman that rats were capable of developing a mental map of the environment. The whole idea of a mental map goes against the behaviorist approach to focus on behavior and reject mental processes. Yet, reinforcement of behavior was insufficient to explain the rats’ behavior. When we engage in spatial navigation it is quite possible that we sometimes learn to go in one way or another at a specific location, but when we know the environment we not only know where to go at that moment, but we understand where we are in relation to other parts of the environment. We have a mental map that guides us to where we want to go. The eventual downfall of behaviorism was not only the consequence of the limitations of reinforcement as the driving force behind our behavior, it was also the result of a new way of thinking about how humans interact with their outside world. The key word here is information. Already during the behaviorist era engineers were concerned about the way information is transmitted through a channel from a source to a receiver. In 1948 Claude Shannon wrote his seminal paper “A mathematical theory of communication” (Shannon, 1948). Information here is seen as the reduction of uncertainty. Obviously, when the content of the message is already known to the receiver there really is no information. So when a coin is flipped, a message containing the result reduces uncertainty. Before the coin flip there are two possibilities, each with a probability of ½, so a message containing the result provides real information. The more uncertainty reducing, or the more surprising the message, the more information is transmitted. Although Shannon’s interest was not human perception, we can readily see how such concepts might apply to how we see the world. When we view a scene and nothing in the scene changes, essentially there is no new information, no surprise. When an object suddenly appears out of nowhere, this is surprising and provides information. In addition to the focus on transmission of information in communication, another important moment for the transition towards a new view of human behavior, was the development of the electronic digital computer in the early 1950s. Here was a machine that receives input, processes this input and generates output. The processing of the input involves some kind of memory storage and a central processor. The way in which the computer processes information became very influential in the way our own mind was viewed. Is this not just what we do? We perceive the world (input), do something with this information (processing) and then use this information to interact with our environment (output). We are essentially information processors. This view meant a radical shift away from behaviorism, towards what would become known as the cognitive revolution. While the behaviorist denied mental processes, the cognitive psychologist was interested in how humans process information. The influence of the computer metaphor can easily be seen in many of the early theories in the field of cognitive psychology. An early example of a classic cognitive model is Broadbent’s (1958) filter model of attention. Important aspects of the model were based on findings from studies on selective listening to speech. In these studies subjects listen to separate messages, one to the left ear and one to the right ear. In terminology inspired by communication theory there were thus two channels through which information was transmitted to the subject, the receiver. It was found that processing information from two channels was extremely difficult. However, when messages were predictable for the receiver, the information content was lower and subjects were better able to follow the messages. Other research demonstrated that when subjects were instructed to repeat the speech from one channel, they were unable to later report the message in the other channel. It appeared that the non-repeated message was not processed in terms of meaning. It was thought that only physical characteristics of this message were received by the subject. This suggested to Broadbent that attention has a limited capacity and when it is directed to one channel, the message from the other channel could be not processed. It was essentially filtered out. Although later research showed that 6 the meaning of part of the message from the unattended channel did get processed to some extent, the idea of attention as a limited capacity process was very influential. Figure 1.1. The original illustration from Broadbent’s (1958) paper. Information from the senses reaches a short-term sensory store, from which it needs to pass the attentional filter before it can be processed further. This filter is selective and determines what information gets processed through the limited capacity channel. From here the processed information is passed on to a memory store and response selection system. Figure 1.1 taken from Broadbent (1958) shows many common features of these early cognitive psychology models. There is input (senses) and output (effectors), there is some central processing, there are different stores where information can be kept, information can be transmitted or copied from one store to the next and there is a limited capacity. The main feature of this model is the selective filter. Information from the senses is kept temporarily in a short-term store (later known as sensory memory, not to be confused with short-term memory), but only a fraction of that information passes the filter so that it can be processed further. Think back for a moment to the behaviorist approach and how different this way of approaching the human mind is. The behaviorists focused on stimuli and responses and rejected mental processes. Broadbent’s model has internal storage and mental processes, features completely foreign to the behaviorist way of thinking. The experimental approach of the cognitive psychologists focused on the examination of these mental processes. To make this cognitive approach more concrete let’s take a closer look at a classic study by Saul Sternberg (1966). While Broadbent’s model focused on an attentional selection of what information gets through the filter, Sternberg’s study examined the nature of the next stage, where information gets stored temporarily in order to carry out some kind of task. Indeed, just like a computer has a random access memory (RAM) for short term storage of data, and a hard drive for long-term storage of data, cognitive models assumed a short-term memory and a long-term memory. The short-term memory, like the computer’s RAM, has a limited-capacity and is used for information currently used to perform a task. Sternberg’s (1966) study examined the nature of this short-term store. If information is held in this short-term store, how is this information accessed? What if there are multiple items held in this short-term store and the observer needs to search through these items? How does the time required to search through these items depend on the number of items held in the short-term store? To examine this Sternberg had subjects remember a set of digits on each trial and then presented a single test digit. The task was to determine as quickly as possible whether the test digit was one of the digits in the memory set. The subject was instructed to pull one of two levers depending on whether the test digit was in the memory set. For example, on a given 7 trial the memory set might be ‘7, 3, 6, 2’. It was assumed that these digits must then be stored in short-term memory. If the test digit that was then presented was one of these four digits, the yes- lever had to be pulled, otherwise the no-lever had to be pulled. On the next trial a new memory set was given, followed by a new test digit. Figure 1.2 shows the average reaction time as a function of the number of items in the memory set. ‘Yes’ and ‘no’ trials are shown separately. Figure 1.2. Mean reaction time as a function of the memory set size adapted from Sternberg’s (1966) study. Regardless whether the test item was in the memory set, reaction times linearly increased as a function of the memory set size. The results show a very clear pattern. The mean reaction time increases linearly as a function of the memory set size. Furthermore, this linear relation holds for trials on which the test digit is in the memory set (yes trials) and for trials on which it is not (no trials). This was interpreted as evidence that all the items in short-term memory are serially scanned before a decision is made. In other words, there is an exhaustive search of the memory set. If this serial search had been terminated once the test digit had been found in the memory set, then reaction times would have been shorter for yes trials than for no trials. Not only do these data provide insight into the nature of the processing of the items in short-term memory, they also provide information about the processing speed. Since mean reaction times increase by about 40 ms for each extra digit in the memory set, the scanning speed of short-term memory could be estimated at around 40 ms per item. This example of a classic cognitive psychology study demonstrates several fundamental principles of the field. Mental processes exist and can be examined through experimental manipulations. Typically reaction times or accuracy of the different conditions were compared and models were developed about the cognitive architecture. Researchers examined processing stages or stores of information, they examined capacity limits, the code in which information is stored and the way in which information is selected, or transmitted from store to store. Generally speaking cognitive psychologists were not so much concerned about how these processes were implemented in the brain. It was assumed these cognitive processes could be studied purely behaviorally. Later this attitude began to change, not in the least thanks to advances in neuroscientific techniques. Nevertheless, in parallel to the development in experimental psychology, from behaviorism to cognitive psychology, biological psychology went through its own development. In fact, much was already known about the brain and how different brain regions were involved in different functions. A short history of biological psychology Biological psychological is essentially the study of the biological basis of behaviour. Sometimes referred to as physiological psychology, it studies the relation between psychological process and physiological processes, or how psychological processes are implemented in the brain. It focuses on 8 an understanding of the human nervous system and functions of different brain regions, but also on the genetic basis of behaviour. Although it’s roots go back all the way to the ancient philosophical problem of how the mind and body are related (the mind-body problem), the most important historical foundation is no doubt the development of Darwin’s theory of evolution in the 2nd half of the 19th century (e.g. Darwin, 1859; 1874). Evolutionary views did exist before Darwin, but it was the publication of ‘On the origin of species’ in 1859 that truly gave evolution theory the attention it deserved. Within time the ramifications would become clear. For one it became understood that mental processes and brain regions evolved with a specific function. Ultimately this function has to do with the survival of the organism and the chances of reproduction. Each species occupies a specific niche in the environment and its nervous system has evolved to be successful in this niche. For each brain region we can ask what its evolved function is and what this allows the organism to achieve. Additionally our understanding of the evolution of man allows us to view other species from a different perspective. Suddenly research into the nervous system of animals becomes relevant for our understanding of the human nervous system. Indeed in the time of Darwin physiologists had already started to examine the nervous system of other species. Despite the differences between the species, the shared evolutionary past places new meaning on this research. Note that the use of animals to understand human functions in the brain comes with ethical problems. These go beyond the scope of this syllabus but are discussed in Carlson and Birkett’s chapter 1. Here we will focus on two techniques that were used by 19th century physiologists that can be helpful in our understanding of the human nervous system. These techniques are electrical stimulation and experimental ablation. The electrical stimulation technique was based on the finding that electrical signals are transmitted through the nervous system. In the 17th century Luigi Galvani had discovered that the electrical stimulation of a nerve attached to a frog’s muscle caused the muscle to contract. This was not only the case for the peripheral nervous system, but also for the central nervous system. This was demonstrated by Gustav Fritsch and Eduard Hitzig in 1870 when they electrically stimulated different parts of a dog’s cerebral cortex (Fritsch & Hitzig, 1870). They found that the effects of the stimulation depended on the place of stimulation. In fact, stimulating different parts of the cortex just anterior to the central sulcus, resulted in the movement of the dog’s muscles. It was even possible to map out the different parts of the dog’s body on this brain region. Stimulating one part might result in movement of one of the leg muscles, while stimulating another region might instead cause movement in a facial muscle. Indeed, this brain region we now know as the primary motor region. Of course, stimulating other parts of the cerebral cortex could also have an effect, but this is hard to determine with a dog as we do not have access to its conscious awareness. Perhaps the stimulation causes the dog to see something, but how would we know? For this we need to use the same technique in humans. Then we can simply let them describe their experience. However, there is an important reason why this technique was used more in animals than in humans; it requires the removal of part of the skull in order to reach the cerebral cortex for the electrical stimulation. Fortunately, there is one circumstance in which this can be done in humans and that is prior to brain surgery. In order to access the brain for the surgery a bone flap from the skull is temporarily removed. If the patient agrees, this opportunity can be used to stimulate the brain electrically, or to measure electrical activity of neurons. This electrical stimulation of the human cortex was performed by Wilder Penfield, which resulted in the publication of the influential work ‘epilepsy and the functional anatomy of the human brain,’ co-authored with Herbert Jasper in 1954 (Penfield & Jasper, 1954). The surgeries were performed in patients with severe epilepsy in order to control the epileptic seizures. With this technique it was possible to create functional maps of several brain regions, in particular the brain regions involved in motor control and perception. For example, stimulating the cortex just posterior to the central sulcus would result in a touch sensation somewhere on the body, while stimulation of part of the temporal cortex might cause the subject to hear a sound. Penfield 9 even found that stimulating another part of the temporal cortex sometimes resulted in some kind of memory retrieval. Although the electrical stimulation technique is useful for understanding brain functions, it does rely on a form of introspection. If the stimulation does not directly cause a muscle movement, we need to rely on the subject’s report of the experience. This limits its usefulness in animal research. For this reason the experimental ablation technique has been more instructive. In this technique a brain region is surgically removed in order to examine its function. The pioneer of this invasive method was physiologist Pierre Flourens, who developed his technique in the early 19th century by removing parts of the brains of various animals and observing the effects. For example, after removing the cerebellum Flourens discovered that balance and motor coordination were affected, providing evidence for the role of the cerebellum in these motor functions. Unlike with electrical stimulation the experimental ablation method allows us to examine the role of brain regions in various tasks. A good example of this approach is the work by Mishkin and Ungerleider in the early 1980s (e.g. Mishkin, Ungerleider & Macko, 1983). These researchers bilaterally removed either the posterior parietal cortex or the inferior temporal cortex in monkeys and gave them a landmark discrimination task or an object discrimination task (see Figure 1.3). In the landmark discrimination task the monkeys had to learn that food would be placed in the covered foodwell that was closest to an object. Here spatial processing is required to quickly get the food reward. In the object discrimination task the monkeys were first presented with a single object. After the object was removed two objects were placed on top of two foodwells, one of which was identical to the earlier presented object, and the monkeys needed to learn that the food was always placed underneath the new object. For good performance in this task the processing of an object’s identity was required. The results showed that ablation of the posterior parietal cortex impaired performance only in the landmark discrimination task, while ablation of the inferior temporal cortex only resulted in an impairment in the object discrimination task. Therefore it was concluded that the inferior temporal cortex is important for identity processing and the posterior parietal cortex for spatial processing (see Carlson & Birkett, chapter 6). Figure 1.3. From Mishkin et al. (1983). Panel A shows the part of the temporal cortex removed in monkeys (above) and the object discrimination task (below) on which these monkeys were impaired. Panel B shows the part of the parietal cortex (above) removed in monkeys and the landmark discrimination task (below) on which these monkeys were impaired. Experimental ablation studies are performed in animals, so we must always consider how well these findings generalize to humans. Naturally we cannot perform the same procedures on humans out of scientific curiosity. However, the experimental ablation method can be compared to the study of patients with naturally occurring brain damage. When brain regions become dysfunctional as the consequence of a stroke, the effects of this brain damage can tell us something about the function of this brain region. A classic example of this is a patient of Paul Broca who had suffered a stroke that significantly impaired his speech production (Broca, 1861). The patient’s name was Louis Victor 10 Leborgne, but was called ‘Tan’ as this was the only word he could say. The brain damage of Broca’s patient was in the left frontal cortex. Later Carl Wernicke discovered that other patients with brain damage in a part of the left temporal cortex had a different kind of language impairment. Their speech production was fine, but their speech comprehension was disrupted. They could not understand speech and what they themselves uttered also did not make much sense. The brain regions involved were later referred to as Broca’s area and Wernicke’s area. The precise language functions of these and other regions has been studied extensively since the days of Broca and Wernicke (see Carlson & Birkett, chapter 14). It must be noted that the study of naturally occurring brain lesions is complicated by their variability. The lesions typically do not just disrupt the function of a single, neatly delineated brain region. Sometimes only part of a brain region is damaged and other brain regions may also be damaged. Nevertheless, as long as these limitations are taken seriously, the findings from lesion studies are a useful part of the picture. In certain situations a neurosurgeon actually does remove part of the brain. This has for example been done to reduce the severity of epileptic seizures. When these seizures become so frequent and so severe that they significantly interfere with everyday functioning, such extreme procedures have been done. The most famous example is patient HM who had a large part of his medial temporal lobe bilaterally removed in 1953. The consequences of this operation were dramatic. His recent long term memories were lost and he could not longer form new long term memories (see Carlson & Birkett, chapter 13). So much research has been conducted on this one patient, between the original report in 1957 (Scoville and Milner, 1957), until his death in 2008, when we learnt his real name Henry Molaison. He is perhaps the most studied patient in history and has contributed greatly to our understanding of memory. Despite the enormous value of studies with lesion patients and patients with operative brain damage, it must always be realized that there are limitations to these studies that must be taken seriously. First, the precise extent of the damage is often not entirely certain unless this is examined postmortem. Second, if there are more brain regions damaged or some regions are only partially damaged, it is not easy to determine the functions of these brain regions. Third, we must always be careful in generalizing to the general population on the basis of a single case study. Many factors, including genetics, disease history and experience can result in brain differences that complicate straightforward generalization. Because of these limitations, findings must always be related to findings from other patient studies and to the findings from studies using other neuroscientific techniques. It is important to gather converging evidence using multiple neuroscientific techniques. These techniques will be addressed in chapter 2. The speed of information processing Before concluding our historical overview of the field of cognitive and biological psychology we will consider a question that has historically been approached from both a cognitive perspective and from a biological perspective. This question is how quickly information is processed. From a biological perspective the question can be asked how quickly signals get transmitted from neuron to neuron, or from a receptor on the body to conscious awareness in the brain. From a cognitive perspective the question can be asked how long a specific mental process takes. Imagine you are a sprinter anticipating the shot from the starting gun for a 100 m athletics race. You get ready in the starting block and concentrate on hearing the gun fire. You get ready both mentally and physically to start running. What happens from the moment the gun fires until your leg muscles contract and you put pressure on the starting block? After the sound reaches the receptors in your ears the signal is passed on to the parts of the brain responsible for auditory processing. Then this sound needs to be interpreted and if the decision is made that the sound comes from the starting gun, the signal is transmitted to the motor cortex, which controls the movement execution. The signal then needs to 11 be transmitted down to the leg muscles before the movement is initiated. All of these steps take time. This decision is a fairly straightforward one and since sprinters want to start as soon as possible presumably not much time is spent on the decision processes. Any loud noise is likely to trigger them. For elite athletes it takes between 140 ms and 200 ms from the moment the gun fires to the moment pressure is applied to the starting block (e.g. Tønnesen et al., 2013). Of course, organizers of athletics competitions know about this, so any start within 100 ms of the firing of the starting gun is considered a false start. Even though you started after the shot, given the time it takes for all these processes to occur, we can safely conclude that you anticipated the gun fire and did not actually hear the shot before deciding to start if your reaction time is less than 100 ms. If it takes roughly 150 ms for all these processes to occur, how long does each of these processes take and how long does it take in the nervous system for information to travel? The first to examine the question of the speed of neural transmission was Hermann van Helmholtz in the middle of the 19th century. He conducted his research on the dissected calf muscle of a frog and the nerve connected to this muscle. It was already known that applying an electric current to the nerve results in contraction of the muscle. Helmholtz reasoned that the response time of the muscle contraction should depend on where on the nerve the current was applied: the closer to the muscle, the short the reaction time. On the basis of the distance between the points of stimulation and the difference in reaction time Von Helmholtz calculated that the neural transmission speed was about 30 meters per second. Once again the question rises how this compares with human neural transmission. Naturally a different procedure is required here. Von Helmholtz stimulated the skin with a weak electric shock at different parts of the body and required subjects to respond with either the hand or the teeth. Similar to the procedure with the frog Von Helmholtz reasoned that reaction time should be shorter after stimulation on the neck compared with stimulation on the foot. After relating the distance of the stimulation on the skin with the difference in reaction times it was concluded that the speed of neural transmission was about 60 meters per second. An important difference between the measurements in the frog and the measurements in the human subjects is that in the human subjects mental processes are involved. Von Helmholtz understood that attention, concentration and even the memory of what type of response is required, could influence the reaction times. Nevertheless, since these mental processes are involved regardless whether one stimulates on the foot or on the neck, the comparison remains valid. The variability just increases so that more repetitions are required for a reliable estimate. Although the reaction time measurements by Von Helmholtz involve mental processes, they do not provide insights into how long each of these processes take. A method suitable for this next step was developed by Franciscus Cornelis Donders in the 1860s and became known as Donders’ subtraction method. Donders reasoned that the duration of a mental process could be measured by having subjects perform two almost identical tasks, one including the critical process and one without the critical process. For example, let’s say we wish to measure the time it takes to discriminate between two visual stimuli. If we simply present one of these two stimuli and measure the response then we are not only measuring the time needed to discriminate between the stimuli, but we are also measuring the time to perceive the stimuli and the time needed to execute the response. Instead, we give the subject two tasks. In task A, the simple detection task, one of two lights is turned on and the subject simply has to press the same button whenever one of the lights is turned on. In task B, the discrimination task, or more precisely the go/no-go task, one of two lights is turned on and the subject is required to press a button when one specified light is turned on, but not when the other light is turned on. It is important to note that everything is identical in the two tasks except for the requirement to determine which of the two lights is turned on. In modern research this would be with a computer monitor and a keyboard. Let’s say the subject either sees a red circle on the screen or a green circle. In the simple detection task the subject always presses the spacebar, 12 while in the go/no-go task the subject only presses the spacebar when the red circle is presented, but not when the green circle is presented. This would be repeated for several trials. On half the trials the red light is presented and on the other half the green light. The order of the trials is randomized so that the subject cannot anticipate what the color of the circle will be. If we average the reaction times in the simple detection task and we average the reaction times in the go/no-go task we will find that the reaction times are longer in the go/no-go task. The difference in average reaction time is then an estimate of the duration of the discrimination process, or the time needed to determine which of the two circles had been presented. It is interesting to note that Donders’ subtraction method fits quite well in the field of cognitive psychology as it assumes mental processes, or stages, even though it would be another century before this field would take over from behaviorism. It is also quite relevant for cognitive neuroscience as the examination of the role of brain regions typically involves the comparison of brain activity between two tasks that are almost identical except for the inclusion of one specific process. This will be discussed further in chapter 2. Towards a cognitive neuroscience The overlap between biological psychology and cognitive psychology should be apparent in this discussion of the historical foundations of the two fields. Although some have been more interested in the human nervous system and others more in mental processes, the questions are often comparable and one could say that the two fields examine two sides of the same coin. Mental processes are the result of activity in the nervous system. There is no mental activity without brain activity. Think back to our sprinter, getting ready to run the 100 meters. In terms of cognitive processes the sprinter perceives the auditory stimulus and identifies this either as the starting gun or some other noise. After this stimulus discrimination the sprinter decides to execute a motor response in response to the gun shot or to ignore the sound if it is some other stimulus. In this sense our sprinter is like a subject in the Donders’ go/no-go task. There are perceptual processes, central processes and motor processes. The central processes retrieve the relevant response for each identified sound, in particular determining whether the sound is the gun shot or not. All of these steps are accompanied by activity in brain regions from the auditory cortex to the motor cortex. Cognitive neuroscientists argue that our understanding of cognitive processes can benefit from brain research and that our understanding of brain processes can benefit from cognitive research. The integration of the two fields into the field of cognitive neuroscience owes much to certain technological advances. Even during the days of cognitive psychology techniques such as EEG were available, but it wasn’t until the 1960s that these electrophysiological recordings could be related to cognitive processes with the discovery of event-related potentials. Even more important was the development of the brain imaging techniques of PET in the 1980s and fMRI in the 1990s. These techniques are being further developed to this day and even newer and more advanced techniques are being introduced. There is no doubt that the field of cognitive neuroscience is booming and to this day we are learning more and more about cognitive processes and how they are produced by the brain. Now do Quiz 1.1 on Canvas to practice your understanding of Chapter 1. 13 Chapter 2: Approaches in cognitive neuroscience Link Lecture 1 (chapter 1 syllabus): In lecture 1 you learned about the history of the fields of cognitive psychology and biological psychology. Throughout this chapter we discussed methods used in the past to examine fundamental questions of experimental psychology, both from a cognitive perspective as well as from a neuroscientific perspective. The discussion is continued here, but now less from a historical perspective, but more from a methodological perspective. How have researchers studied cognitive processes and the role of the brain in these processes? Link Lecture 2 (Neural and Glial cells, Carlson and Birkett): In lecture 2 you learned about the nerve cells and the action potentials in these cells. This is essentially the language of the brain: neurons producing action potentials. Any cognitive process is accomplished through neurons producing action potentials. Here you will learn how this neural activity is measured, directly or indirectly, and how we can relate this to cognitive processes. Reaction time methods In order to understand the methods used in cognitive neuroscience we need to start simple. We need to first understand some basic principles of behavioral methods. Once we understand those we can include neural aspects and ultimately try to understand how cognitive processes are represented in the brain. As we do this we will make the methods concrete and illustrate how they have been used to examine some fundamental research questions. In chapter 1 we addressed some historical questions that demonstrated how important the factor time is. Cognitive processes take time and that time can be measured. Whether it is Sternberg who measured the speed of scanning short-term memory, or Donders who measured the duration of a specific cognitive stage, time is an important factor. Simply measuring how long a task takes and comparing this to how long a slightly different task takes can tell us something about what is going on in the mind. Consider what happens when a subject directs attention somewhere in a visual scene. In chapter 1 we introduced Broadbent’s classic view of attention as a selective filter. That is, attention selects what gets further processed. So what effect does directing attention to something or to some place have on performance in a task? Think back to our example of the athlete waiting for the firing of the starting gun. The runner can either pay attention before the firing of the starting gun or just wait for it to happen. If we were to measure the reaction time to the firing of the gun we would surely find that the reaction time is shorter when the athlete is paying attention. What now if the athlete is expecting to hear the gun fire to his right, but in actual fact the gun fires to his left? What effect does this spatial attention have on performance? It was this fundamental question that was addressed in the classic study of Posner (1980). Just like an athlete can attend to the firing of the starting gun, a subject in an experiment can respond with a keypress to the appearance of a stimulus on the screen. This is one of the simplest tasks that can be performed. It does not require much cognitive processing. Posner examined what effect the allocation of attention could have on the response. There have been many variants of this procedure, which we refer to as the Posner cueing task, but here we will focus on the central arrow condition of the task. See Figure 2.1 for the display sequence of the central arrow condition from Posner (1980). 14 Figure 2.1. The display sequence of the central arrow condition of Posner (1980). On top is shown the timing of the display sequence in milliseconds. Below this from top to bottom are the cue displays and targets displays of neutral trials, valid cue trials and invalid cue trials. After a variable intertrial interval (ITI) an arrow cue is presented on the valid and invalid cue trials, while on the neutral trials a fixation cross is shown instead of the arrow. Then the target object (detection stimulus) is shown to the left or right of the fixated object. The task is to press a key as quickly as possible once the target object appears. On valid cue trials (the majority of trials) the target object is presented at the indicated location, while on invalid cue trials (the minority of trials) the target object is presented at the opposite location of the indicated location. In this condition subjects view a screen and look at a small cross (the fixation cross) in the center of the screen. After some time the cross is replaced by an arrow pointing to the left or to the right. After a variable delay of around a second a target object appears to the left or to the right of the arrow and subjects are required to press a key as quickly as possible upon its appearance. This is repeated for many trials and the location of the target and the direction of the arrow are varied between trials. So far this seems to have nothing to do with attention; the subject just responds to a target. However, the crucial manipulation is that on the majority of trials (say 80 %) the target is in fact presented at the location indicated by the arrow (i.e the cued location). Now how do you think the subject will respond to this? If the subjects follow the instruction and try to respond as quickly as possible, their attention will shift to the location indicated by the target. The subjects are not allowed to move their eyes, but attention can shift without an actual movement of the eyes. The consequence of this is that reaction times are shorter when the target is indeed presented at the cued location than when it is presented at the uncued location. The task also typically includes control trials, on which no central arrow is presented so that the subject has no expectation of where the target will appear (i.e. the neutral condition or control condition). Typically reaction times are longer when the target is presented at the uncued location (i.e. invalid trials) compared to the neutral condition and shorter when the target appears at the cued location (valid trials) compared to the neutral condition (see Figure 2.2). 15 Figure 2.2. Typical results from the central arrow condition of the Posner cueing task. Mean reaction times are longer on the invalid cue trials than on the neutral trials and shorter on the valid trials than on the neutral trials. Here it is assumed that 70 % of the trials are valid cue trials, 20 % are invalid cue trials and 10 % are neutral trials. What does this tell us? Posner (1980) interpreted the results as evidence that attention shifts from location to location in order to detect the target. So when a predictive arrow cue is presented the subject shifts attention to the cued location. When the target is indeed presented at the cued location this presence of attention helps the subject to detect the target. However, when the target is presented at the uncued location, attention is first shifted to the uncued location in order to detect the target. This shift of attention takes time, which explains the difference in reaction times between valid trials and invalid trials. In other words attention aids detection. As per Broadbent’s model attention is a selective filter for further processing. So the spatial allocation of attention affects reaction times in the Posner cueing task, but can the effect of attention also be found in neurophysiological measures? This question will be addressed in the next section, but first we will discuss how attention is involved in another task and how his can be measured through reaction times. This task is the visual search task. In the visual search task subjects are instructed to determine whether a specified target object is presented on the display. Several objects are simultaneously shown and on half the trials the target is present and on the other half it is absent. Subjects are required to press one key when the target is present and another key when the target is absent. The crucial factors that are manipulated are the nature of the other objects (the non-targets) and the number of objects presented. Two types of visual search are distinguished, feature search and conjunction search. In the feature search task the target can be distinguished from the non-targets on the basis of a single feature, while in the conjunction search task all features of the target are shared by at least one of the non-targets. See Figure 2.3 for example displays. 16 Figure 2.3. Example displays from a typical visual search task. The left panel illustrates the feature search task where the target (red vertical bar) can be distinguished from the non-targets on the basis of a single feature (here: color). The right panel illustrates the conjunction search task where the target (red vertical bar) cannot be distinguished from the non-targets on the basis of an individual feature. That is, some of the non-targets are also red and some of the non-targets are also vertical. These panels show target present trials. Note that on half the trials the target is absent and the subject is instructed to determine whether the target is present or absent. In the visual search task researchers examine whether reaction times increase as a function of the number of objects (i.e. the set size). What is typically found is that reaction times only increase significantly as a function of set size in the conjunction search task, but not in the feature search task (see Figure 2.4). Figure 2.4. The typical pattern of results in the feature search task (left panel) and the conjunction search task (right panel). Mean reaction times increase as a function of set size in the conjunction search task, but not in the feature search task. This actually makes a lot of sense when one considers the search tasks shown in Figure 2.3. In the left panel the target pops out. Adding more non-targets really makes no difference. However, in the right panel the target does not pop out and the subject really has to search for it. Specifically, in the conjunction search task attention shifts from object to object until the target is found. Adding more non-targets simply means that more objects need to be inspected. In the feature search task, the finding that reaction times hardly increase as a function of set size is interpreted as evidence that attention is not required to find the target. So in this task reaction times can tell us something about the involvement of attention in the visual search task. As we observed in the discussion of the Posner cueing task, these purely behavioral reaction times are surely meaningful, but from a neuroscientific perspective we can ask whether more insights can be gained from examining how these tasks are performed in the brain. The following sections therefore introduce some brain methods and we will return to the earlier discussed Posner cueing task to illustrate how these methods can be used. 17 Event-related potentials In lecture 2 (neutral and glial cells) the action potential of neurons was discussed. When neurons are stimulated past a certain threshold they will deliver an action potential. The activity of a neuron can be measured by the number of action potentials delivered per second. Of course at any given moment electrical activity occurs throughout much of the brain and this activity can be measured on the scalp through electroencephalography (EEG). See Figure 2.5. Figure 2.5. A subject wearing a cap on which electrodes have been attached for electrophysiological measurement of brain activity, a method known as electroencephalography (EEG). When examining neural activity related to cognitive processes, what the researcher is really interested in is how this EEG activity changes as a function of these cognitive processes. The difficulty lies in the fact that so much of this electrical activity is random noise, or at least not directly related to the cognitive processes related to task performance. This noise needs to be filtered out in order to examine the activity related to cognitive processes. Fortunately, since this activity is random and changes from trial to trial, all one needs to do is to average the activity across many trials and then the noise is ultimately averaged out, leaving only the electrical activity related to the cognitive processes. See Figure 2.6 for an illustration of this procedure. Figure 2.6. In order to average out the random noise the EEG segments are time-locked to a certain event. For example, if a stimulus is presented on each trial we can take the EEG segments starting from the moment the stimulus is presented. We do this for many trials (N stands for number of trials), so for stimulus 1, stimulus 2, etcetera, up until stimulus N. On any given trial the EEG activity that occurs in response to the stimulus is overshadowed by the noise, but after averaging all trials 18 the noise is removed and we see how the brain responds to the stimulus. Note that the scale on the y-axis in the right panel is much smaller than in the left panels as the activity in response to the stimulus is only a fraction of the total EEG activity. After the noise is removed we can observe the so-called event-related potential (ERP) that tells us something about the cognitive processing occurring. The cognitive activity in response to a given event can be examined through the event-related potential (ERP). Within the ERP waveform we can identify certain components and it is these ERP components that inform us about the cognitive activity. These components are named for whether they represent a negative or positive deflection of the waveform (N or P) and are given a number. This number often reflects the order in which they occur (i.e. P1 occurs before P2) or it reflects the approximate time at which it typically occurs (i.e. N400 or P600 occur at roughly 400 ms or 600 ms after stimulus presentation). Yet other ERP components have received a name that reflects its function. Whether an ERP component is negative or positive is not so much meaningful, but rather it is the amplitude of the component that is examined and/or the delay with which it occurs. Since the ERP waveform is time-locked to the event we can typically say that the later the component occurs the more complex the cognitive process is that it reflects. An early component such as the P1 represents the detection of the stimulus, while a much later component like the N400 processes semantic information of the stimulus. As an illustration of the use of ERP measurements we will return to consider a variant of the earlier discussed Posner cueing task. In the selective attention experiment from Luck and Ford (1998) subjects are instructed to attend to a location to the left or right of the central fixation location. The attended location is fixed for a whole block of trials and is switched to the other location for the next block of trials. The task is to detect a specified target object, which is presented amidst the presentation of a sequence of non-target objects, that are presented at the attended and unattended locations. See Figure 2.7 for an illustration of the stimulus display in the selective attention experiment. Figure 2.7. Stimulus display of the selective attention experiment of Luck and Ford (1998). Subjects keep their eyes of the central fixation cross and attend to the location to the left or right of fixation. They are required to detect a target object presented at the attended location and to ignore non-target objects that are presented at the attended and unattended locations. Similar to the central arrow condition of the Posner cueing paradigm, subjects are attending to one of the locations and the target appears at the attended or at the unattended location. In the purely behavioral version of the task the finding is that reaction times are shortest when the target is presented at the attended location, but in Luck and Ford (1998) the question is what effect attention has on ERP components. Figure 2.8 shows the pattern of results. 19 Figure 2.8. ERP results from the selective attention experiment of Luck and Ford (1998). The solid line represents the ERP waveform in response to objects presented at the attended location and the dashed line represents the ERP waveform in response to objects presented at the unattended location. In Figure 2.8 it can be seen that attention has an effect on the P1 and N1 components, but not on the C1 component (named C1 as it is simply the first component). In other words, the attentional modulation does not occur at the earliest processing stage, but does have its effect later on. This attentional modulation can be interpreted as an enhancement of the processing of the object presented at the attended location. Consistent with the reaction time results, attention improves processing. The selective attention experiment of Luck and Ford (1998) is an example of a relatively early effect on ERP components as it occurs between 100 and 200 ms. If we consider more complex cognitive processes such as language, later components are examined. Our second example illustrates how the N400 and P600 components are related to language processing. An important concept related to language processing is predictive processing. This will be addressed in more detail in Chapter 3, but here we will use this concept to illustrate the use of ERP components. When we read text we try to make sense of the text. During the reading of a sentence this understanding is provisional as only part of the sentence has been read and one must interpret both the meaning of the words and the grammatical structure. However, this provisional understanding of the text does allow us to predict what comes next. For example, when we read “I need my umbrella, because it is …. “, the word “raining” is already being activated before it is read. Another concept related to predictive processing is surprise. That is, when something occurs that significantly deviates from the prediction, there is surprise, which provides a signal to interpret the cause for this surprise. In Osterhout, McLaughlin and Bersick (1997) subjects read sentences that did or did not contain a surprise. The surprise could be semantic (i.e. meaning), syntactic (i.e. grammatical) or both. See Figure 2.9 for the corresponding ERP patterns. 20 Figure 2.9. ERP waveforms from Osterhout et al. (1997). The solid lines are from sentences without a surprise, while the dotted lines contain some sort of surprise. In (A) the surprise is semantic, in (B) it is syntactic, and in (C) it is both semantic and syntactic. The ERP waveforms reveal that the surprising sentences result in some sort of deviation from the normal pattern of the sentences without a surprise. When the sentence contains a semantic surprise the N400 amplitude is affected and when it contains a syntactic surprise the P600 (sometimes referred to as the SPS or syntactic positive shift) is affected. Thus, it takes longer to process the syntactic anomaly than the semantic anomaly. The ERP examples demonstrate that there are components that can be affected by various cognitive processes. Attentional effects typically occur before more complex effects such as those related to semantics and syntax. It is important to realize that the analysis of these ERP components offers good insights into the timing of cognitive processes. In other words, the temporal resolution is very high. They can tell us quite accurately when a certain manipulation affects information processing. However, the spatial resolution is not so strong. These measures do not give us accurate information about where the brain activity is coming from. Although source localization can be done by comparing the waveforms from different electrodes, this information is not as accurate as can be gathered from other methods, such as the brain imaging methods of PET and fMRI. Most ideally findings from ERP research are compared with findings from studies using these other methods so that we can paint a fuller picture and strive for converging evidence. 21 Brain imaging methods PET and fMRI are two brain imaging methods that are quite similar. In contrast to ERP measurements they have a poor temporal resolution, but a good spatial resolution. This means that they are good methods to examine where brain activity occurs, but not to accurately examine when this activity occurs. While ERP measurements are direct measurements of brain activity, PET and fMRI are indirect measurements. They do not measure the electrical activity, but rather the blood flow in the brain. The principle behind this is that when a brain region is active, this activity is followed by increased flow of oxygenated blood to this brain region. The way in which this is measured differs between PET and fMRI. While PET uses a radioactive tracer that is injected in the bloodstream, fMRI measures the changes in magnetic properties as the result of the presence of oxygen in the blood. Both PET and fMRI need to deal with the fact that there is always some activity in the brain. So the finding that a certain brain region is active, does not really mean much unless one can compare this activity with the activity in some control condition. That is, it is never about how active a brain region is, but rather how active a brain region is in one condition compared to another. To illustrate this procedure we will discuss the PET study by Petersen et al. (1988). This study examined which brain regions are involved in different cognitive processes related to reading. Consider for a moment what happens when you read a word like ‘cake’? What cognitive processes lead to the retrieval of the meaning of this word? Well, first perceptual processes need to occur to identify the features of the letters. Once the letters have been perceived the word can be recognized. Finally the meaning of the word is retrieved from memory (see Figure 2.10). Figure 2.10. Cognitive processes involved with reading the word ‘cake’. Figure from Ward (2020). See text for details. We could give the subjects a task that requires the processing of meaning. For example, we could ask them to generate an action for the word. So for ‘cake’ they could say ‘bake’ or ‘eat’. To do this they would need to retrieve the meaning of the word ‘cake’. However, if we were then to examine the brain activity during this task we would see activity in many different regions. Some brain regions would be involved in perceptual processing, others in word identification and yet others in the retrieval of the meaning of the word. Unfortunately we would not know which brain region is involved in which cognitive processes. To avoid this problem the authors created a control condition for each cognitive process of interest. In the first experimental condition subjects were required to passively view a word. They did not have to do anything with it. In the first control condition subjects viewed a fixation cross on the screen. Now both of these conditions involve some form of perceptual processing, but only in the experimental condition is there word recognition. So when the brain 22 activity from the fixation cross condition is subtracted from the brain activity in the passive word viewing condition then the resulting difference indicates which brain region is involved in word recognition. To examine which brain regions are involved in saying words the authors followed a similar procedure. Now the experimental condition was reading aloud a written word and the control condition was the passive viewing of the same word. Now the only difference between the experimental condition and the control condition is the articulation of the word. So subtracting the activity in the passive word viewing condition from the activity in the reading aloud condition revealed which brain region is involved in articulating words. Finally, to examine which brain region is involved in retrieving the meaning of a word from memory the authors used the earlier described task of providing an action for the word. Comparing the brain activity in this condition with the word reading condition provided the brain region for retrieving word meaning. See Figure 2.11 for the brain regions that are left after subtracting the activity in the control conditions from the activity in the experimental conditions of Petersen et al. (1988). Figure 2.11. In order to determine which brain region is involved in recognizing words (purple), saying words (brown) and retrieving the meaning of the words (green) Petersen et al. (1988) subtracted the brain activity of a suitable control condition from the brain activity of the experimental conditions, such that the difference could be related to the critical cognitive process. See text for details. Figure from Ward (2020). Note that in these conditions from Petersen et al. (1988) the experimental conditions involve the critical cognitive process, while the control condition is as similar as possible, but does not involve the critical cognitive process. This is a common procedure in brain imaging research, whether the method uses PET or fMRI. It is a nice way of examining which brain regions are involved in certain cognitive processes. However, brain imaging is not only about discovering where in the brain something happens. Once we learn about the roles of different brain regions we can also use this knowledge for other questions. We can compare different groups of subjects (e.g. a patient group and a control group) and examine whether there are differences in the extent to which certain cognitive processes are used by these groups. We can also examine whether certain cognitive processes are used more in one task than in another. All this of course relies on our knowledge of which brain regions are involved with these cognitive processes. As an example consider the question whether perceptual processes are used in mental imagery. Imagine closing your eyes and forming a mental image of a tree. When you do this there is no actual visual input. You see the tree with your mind’s eye as it were. How is this similar to or different from actually looking at the tree? 23 Well, we know that the occipital lobe is for a large part involved with perceiving objects, as are certain regions of the temporal and parietal cortex. This will be discussed in more detail in the chapters on perception, but here we will use the question of the relationship between mental imagery and visual perception as our second example of the use of brain imaging. The study we will use for this is the study by Ganis et al. (2004). The authors conducted an fMRI study to investigate the brain activity during mental imagery and compared this with the brain activity during visual perception. As always with these studies the conditions were made as similar as possible with the critical factor as the only difference. Specifically, subject were either instructed to imagine a certain object or they were shown a faint image of the object. Whether they were imagining the object or looking at the object they received questions about the object. So for example, if the object was a tree they might be asked whether the tree is wider than it is tall. See Figure 2.12 for an example of this method. Figure 2.12. In Ganis et al. (2004) subjects were instructed to either imagine a tree or they would perceive a dim image of the tree. Then they would be asked a question about the image (imagined or seen). If mental imagery requires perceptual processes then a large degree of overlap is expected between the brain activity in the perception and imagery conditions. Of course, there must be some difference between the two conditions. After all, imagery is not the same as perception, however strong the similarity might be. See Figure 2.13 for the brain imaging results of Ganis et al. (2004). Note that in this figure the brain activity is shown relative to the baseline activity between trials when neither the perception task nor the imagery task was being performed. 24 Figure 2.13. Results from Ganis et al. (2004). The left two columns represent the brain activity in the perception and imagery conditions respectively relative to a baseline condition. The third column from the left is the difference between these conditions. The rightmost column shows from where in the brain the activity is taken. The results from Figure 2.13 show a large degree of overlap in the brain activity in the perception condition and the imagery condition. Only in the most posterior brain regions of the temporal cortex and in the occipital cortex was there a difference of any significance (see bottom-right panel). There was more activity in these regions in the perception condition than in the imagery condition. This is of particular interest as these regions are involved with visual processing. So this suggests that visual processing regions are more involved in the perception condition than in the imagery condition. This can hardly come as a surprise given the nature of the perception task. However, it is noteworthy that there actually also was activity in these visual processing regions in the imagery condition, just not as much as in the perception condition. This suggests that visual processes might also be involved in the imagery condition. So here our understanding of the functions of these brain regions can contribute to our understanding of which processes are involved in the task. Although this is quite meaningful, we do need to be cautious with interpreting these results. What precisely are these brain regions doing in these tasks? In Ganis et al. (2004) we could ask whether the activity in the occipital lobe is actually critical for performing mental imagery. Unfortunately brain imaging cannot provide this information. For this we need to turn to the final method of this chapter, Transcranial Magnetic Stimulation (TMS). Transcranial Magnetic Stimulation (TMS) TMS is a method that allows us to examine whether certain brain regions are causally involved in certain cognitive functions. It allows us to disrupt the functioning of a brain region to examine what effect this has on task performance. A large coil is placed over the critical brain region, which delivers a magnetic pulse to the brain. See Figure 2.14. This interferes with the electric activity within the brain region, so that the brain region can no longer function properly. 25 Figure 2.14. A subject undergoing Transcranial Magnetic Stimulation (TMS). A coil is placed over the scalp in order to deliver a magnetic pulse to the brain region of interest. See text for further details. There are other ways in which TMS is used, but temporarily disrupting the performance of a brain region is kind of like creating a temporary lesion. Fortunately it is a non-invasive method and quite harmless. Its effects on the brain are also very short-lived. In fact, it needs to be timed right, otherwise it will have no effect on task performance. Often the magnetic pulse is repeated (rTMS or repetitive Transcranial Magnetic Stimulation) to prolong the effect, otherwise task performance might not be affected. TMS does have in common with lesion studies that it allows us to examine the function of brain regions and especially whether the brain region is really needed to perform a certain task. Let’s return to the question of the previous section to illustrate the use of TMS. Is the occipital lobe needed to perform mental imagery. In other words, are visual processes critically involved in the task? This was the purpose of a study by Kosslyn et al. (1999). To examine this question subjects saw a screen containing four quadrants containing a number of bars that varied in number, orientation, length and width. See Figure 2.15 for an illustration of the stimulus display. Figure 2.15. An illustration of the stimulus display used in both the Perception and Imagery conditions of Kosslyn et al. (1999). See text for details. The four quadrants were numbered and on each trial subjects would hear two quadrant numbers and then an attribute, such as length, width or number, and then had to compare the two quadrants on this attribute. Critically, in the Imagery condition the stimulus was removed before the comparison was made so that they had to form a mental image of the stimulus display. In the Perception condition the stimulus display remained present so that no mental image needed to be constructed. On some of the trials of both the Imagery and the Perception condition rTMS was 26 applied while on other trials sham TMS was applied. Sham TMS can be considered a control condition for TMS as everything is kept the same, including the positioning of the coil and even the sounds made by the apparatus, in order to prevent the subject from knowing whether magnetic pulses are actually being delivered. The rTMS was applied to a part of the occipital lobe known to be involved in early visual processing. The rationale behind this procedure is that if this part of the occipital lobe is not only involved in perception, but also in imagery, then performance in the Imagery condition should be impaired in the real TMS condition relative to the sham TMS condition. The results are shown in Figure 2.16. Figure 2.16. Results from the TMS experiment of Kosslyn et al. (1999). Shown are the mean response times of the individual subjects (numbers in the figure) in the Perception and Imagery conditions after real TMS and sham TMS. The results of the Kosslyn et al. study are quite clear. Not surprisingly TMS did significantly impair performance in the Perception condition. This is to be expected as the disrupted occipital region is known to be involved in early visual perception. More importantly, TMS also affected performance in the Imagery condition. So disrupting a part of the occipital lobe also impairs the ability to perform mental imagery. This is clear evidence that visual perceptual processes are also involved in mental imagery. Note that this evidence goes beyond what can be shown by brain imaging as the TMS evidence shows that this region is not just activated in this task, but that this activity is really needed to perform this task well. Kosslyn et al. also performed a PET study with similar tasks and showed, consistent with the TMS findings, that the area that was targeted in the TMS experiment, was activated in the PET study in the Imagery condition. This provides converging evidence: brain imaging studies show which regions are activated in a task, while TMS can show whether this activity is actually causally related to the task performance. Converging and conflicting evidence In the previous sections we have covered several different methods to examine cognitive processes in the brain. Some methods such as PET and fMRI are very good at informing us which brain regions are activated in different conditions of different tasks and to what extent they are activated. TMS is a technique that is ideal to examine whether a certain brain region is causally involved in a certain task. 27 ERP measurements allow us to how cognitive processes are affected by certain factors and are particularly good at revealing the time course of these cognitive processes. We could add to these methods some methods from Chapter 1 such as lesion studies and ablation studies that also provide information on the function of certain brain regions. We could also add to this the method of single cell recordings, which will be addressed in Chapter 3. This method allows us to examine the extent to which individual neurons are active in a certain task. All of these methods have their advantages and their disadvantages, but what we really need is converging evidence. Interpretations results from different methods should converge towards the same conclusions, or at least be compatible with one another. Take the study of Kosslyn et al. (1999) from the previous section. Here a brain region was activated in an imagery task, as shown by PET, and when applying rTMS to this brain region performance in the imagery task was impaired. This is converging evidence. Sometimes evidence from different methods appears to conflict at first glance. Often these cases provide a puzzle and after solving this puzzle we are even closer to understanding the underlying processes. As an example we will return to the ERP study of Luck and Ford (1998). An important finding from this study was that the allocation of attention did not affect the earliest ERP component C1, but did affect the following components such as P1 and N1. This suggested to the authors that attention has no effect on the earliest visual perceptual processes. These earliest visual processes are thought to occur in brain region V1 (i.e. the part of the visual cortex where visual information first arrives). Well, this is something that can of course be checked with brain imaging. This was in fact done by Martinez et al. (1999). In this study both fMRI and ERPs were examined in an attention task. Subjects were instructed to attend to a location indicated by a central arrow and to detect a target stimulus at that location. Streams of stimuli were presented at the attended location and at the opposite unattended location. This allowed the authors to examine the response of the brain to stimuli presented at both attended and unattended locations. Consistent with the findings from Luck and Ford (1998) attention did not affect component C1, but did affect the later components. However, brain imaging analyses did reveal that attention did have an effect on brain region V1. That is, when subjects attend a stream of stimuli presented at the attended location this resulted in greater activity in V1 than when the stimuli were presented at an unattended location. These results appear to conflict as component C1 is thought to represent processing in V1, yet it is unaffected by attention. Then why does attention affect the brain activity in V1 when measured by brain imaging? A key to the solution of this puzzle is to understand the different strengths and weaknesses of these methods. ERPs have a really high temporal resolution but a low spatial resolution, while fMRI has a really high spatial resolution but a low temporal resolution. So the C1 component really represents extremely early (within 100 ms) visual processing, but the activity found in fMRI analyses is the result of activity occurring over several seconds. It is thought to be the result of feedback from higher visual regions back unto V1. This of course does not affect the processing within the first 100 ms, but later on once higher visual processing regions have processed the information, these regions could modulate the activity in V1 as a form of feedback. This idea will be discussed in more detail in chapters 3 and 4. To conclude, whether or not results from different methods converge, they should always be considered together to form a full picture of the cognitive processes occurring in the brain. Comparing methods with a good spatial resolution with methods with a good temporal resolution allows us to take advantage of the strong features of different methods. Of course, ideally we would use a method that has both a high spatial resolution and a high temporal resolution. Such a method in fact does exist and is called single-cell recording. Chapter 3 will discuss this method. Despite its high spatial and high temporal resolution it does have its drawbacks. For one, it is very invasive as it relies on implanting electrodes in the brain, which means removing part of the scalp. It is also a technique that is usually performed on other mammals, so the question always remains how well the 28 results generalize to humans. Nevertheless it is yet one additional method that can be very informative in painting a full picture as we strive for converging evidence. Now do Quiz 2.1 on Canvas to practice your understanding of Chapter 2 29 Chapter 3: From Neuron to Cognition Link Lecture 3 (chapter 2 syllabus): In lecture 3 you learned about different methods that are used in cognitive neuroscience to examine the involvement of brain regions in cognitive processes. The current chapter presents one additional method, single cell recordings. This method allows us to examine the activity of individual neurons. We will address the question of what this activity actually means. Link Lectures 2 and 4 (Neural and Glial cells; Neurotransmission, Carlson & Birkett): In lectures 2 and 4 you learned about the structure of the neuron, the action potential and neurotransmission. The current chapter will focus on the action potentials of neurons, the measurement of the activity of these neurons (i.e. single cell recordings) and move from there to the question of how this activity relates to cognitive processes. Measuring single cell activity. In Chapter 2 we discussed methods that directly or indirectly measure the activity of the brain. The brain imaging methods of fMRI and PET examine the activity of different brain regions. This allows us to see the big picture, but what we see in these brain imaging methods is, albeit indirectly, the consequence of the activity of individual neurons. From this perspective neurons are the building blocks of the brain. So to understand brain activity and how this relates to cognition we also need to understand the activity of individual neurons. The method for this is known as single cell recordings. In order to record the activity of individual neurons an electrode needs to be placed in the brain. To do this a part of the scalp needs to be removed for the microelectrode to be inserted. This is of course a very invasive procedure, and one that under normal circumstances is not performed on humans. So typically other species such as monkeys are used for this (see Figure 3.1a). This is of course a disadvantage, because the question always remains how well the results generalize to humans. Furthermore, some cognitive processes such as those related to language can obviously not be studied in other species. Therefore findings from single cell recordings should be interpreted with caution and seen as part of the whole picture as we consider converging evidence from multiple methods. Once we record the activity of the neurons, what is it that we are interested in? In Carlson and Birkett it was explained that action potentials operate on an ‘all or nothing’ basis. That is, the magnitude of the action potential is always the same (see Figure 3.1b and c). So when we measure the action potential its magnitude is not of interest. Therefore, if a neuron becomes more involved in the activity in the brain, it is not the magnitude of the action potentials that is measured, but rather the frequency with which these action potential occur (Figure 3.1d). How many action potentials, or spikes, as they are often referred to, are delivered by the neuron can inform us about the function of this neuron and its role in ongoing cognitive processing. This frequency is known as the firing rate of the neuron. However, neurons have a base rate of activity. A single action potential does not necessarily mean much. Measurements of neural activity, single cell recordings, are always done as a comparison between different conditions. It is the firing rate of a neuron in one condition compared to its firing rate in other conditions that can provide useful information. 30 Figure 3.1 (From Goldstein, 2011). (a) The typical setup of single cell recordings. The axon’s membrane potential (recording electrode) is compared to the charge outside the axon (reference electrode). (b) The resting potential before an action potential is produced. (c) The change in the membrane potential as a function of time when an action potential is produced. (d) Counting the number of action potentials per time unit, also known as the firing rate of a neuron. We will illustrate this method by returning once more to our question of the effects of attention on brain activity. In Chapter 2 we saw evidence from Luck and Ford (1998) that attentional allocation does not have an effect on the earliest ERP component C1, but did have an effect on the subsequent ERP components. Since it is thought that the C1 component is produced by the early visual cortex, an interesting question is whether single cell recordings of the early visual cortex are affected by the allocation of attention. Given the findings on the C1 component no such effect of attention on the firing rates of neurons in the early visual cortex is expected. This would provide further converging evidence. Luck and Ford (1998) relate their findings to an earlier study (Luck et al., 1997), in which single cell recordings of the visual cortex of the macaque were made, in a task in which the macaques were trained to attend to one of two objects presented simultaneously on the screen. The question was what effect this allocation of attention would have on the activity of neurons in V1, the early visual cortex, and V4, a region higher up in the visual processing stream. The results are shown in Figure 3.2. The objects are presented at 0 ms, so this figure nicely shows the early baseline activity of the neuron, followed by an increasing firing rate (spikes per second, or the number of action potentials per second). For both regions of the visual cortex the firing rate of a neuron responding to an attended object is compared with the firing rate of a neuron responding to an unattended object. This comparison represents the effect of attention. As can clearly be seen in the figure, the firing rate is greater for attended than for unattended objects in V4, while no such effect of attention is present 31 in V1. This again shows that attention has no effect on the earliest visual processes, but does affect later visual processes. This forms compelling converging evidence with the findings from Luck and Ford (1998). Figure 3.2. Data from Luck et al. (1999), reprinted by Luck and Ford (1998). The top panel shows that the firing rate of neurons in V4 is greater for attended objects than for unattended objects. The bottom panel shows that this is not the case for neurons in V1. Attention does not affect the firing rate of the neurons in V1. In this example of the use of single cell recordings the question was asked whether the activity of neurons is affected by an experimental manipulation, in this case where attention is directed. Together with evidence from other methods this study has shown that attention can increase brain activity, and as such enhance the processing of an object. Specificity coding versus distributed coding Our second example of the use of single cell recordings focuses on the question how specific a neuron’s response can be. Imagine we present one object, say a table, and we see that one of the neurons from which we are measuring significantly increases its firing rate. Then we present another object, say a plant, and we see that this has no effect on the firing rate of the neuron. What does this mean? Well, not that much really. We would need to repeat this with more presentations of tables and plants, and preferably many other objects, and we would also need to measure the response of many other neurons for a fuller picture. Indeed, something like this was done in the study by Quiroga et al. (2005). This single cell recording study was actually performed on human subjects. Even though the method is very invasive there are circumstances in which they can be performed on humans. In this case the subjects were epileptic patients who had electrodes implanted in order to trace the focus of the epileptic seizures. This allowed the authors to examine the specificity of the response of neurons in the medial temporal lobe. Figure 3.3 shows the response of one of these neurons to a wide range of pictures. 32 Figure 3.3. The response of an individual neuron to a selection of the pictures shown (from Quiroga et al., 2005). For each picture the figure shows the number of spikes (i.e. action potentials) as a function of time. Each blue dot is an individual spike and the different lines of blue dots represent different presentations of the same picture. The red histogram shows the number of spikes summed across the successive presentations of the pictures. Note the strong response to pictures of Jennifer Aniston compared to the other pictures. This neuron appears to have a very specific response. It has a high firing rate for different pictures of Jennifer Aniston, but hardly responds to the other pictures. What does this mean? It is tempting to call this neuron the Jennifer Aniston cell. Could it be that this single neuron is responsible for representing Jennifer Aniston? Perhaps its activity simply means that the patient has recognized Jennifer Aniston. The authors are quick to add caution to their findings. It could easily be that other neurons also respond to pictures of Jennifer Aniston, but these were simply not measured. Furthermore, it could be that the neuron would also respond to other people or objects. After all, the set of pictures used was limited. Another important point to make is that not all neurons that showed a response responded as selectively as the neuron from Figure 3.3. In fact, roughly 40 % of the responsive neurons responded strongly to only a single person or object. The other neurons’ responses were more complex. However we are to interpret these results, the question of how information is actually represented in the brain is a very fundamental one. The two most extreme ways in which information can be represented are through specificity coding and distributed cod

Syllabus Cognitive Psychology Chapters 1-9 PDF

Document Details

Tags

Related

Summary

Full Transcript

Upgrade to continue