Sensation and Perception Chapter 1 PDF

METHOD 1: THRESHOLDS What is the faintest sound you can hear? How would you know? What is the loudest sound you can hear? This last question is not as stupid as it may sound, though it could be rephrased like this: "What is the loudest sound you can hear safely or without pain?" If you listened to...

METHOD 1: THRESHOLDS What is the faintest sound you can hear? How would you know? What is the loudest sound you can hear? This last question is not as stupid as it may sound, though it could be rephrased like this: "What is the loudest sound you can hear safely or without pain?" If you listened to sounds above that limit, perhaps by blasting your music too enthusiastically, you would change the answer to the first question. You would have damaged your auditory system and be unable to hear the faintest sound that you used to be able to hear. Your threshold would have changed (for the worse). How would you measure that threshold? As we'll learn in this chapter, a variety of methods are available for measuring just how sensitive your senses are. METHOD 2: SCALING---MEASURING PRIVATE EXPERIENCE When you say that you "hear" or "taste" something, are those experiences---what the philosophers call qualia (singular quale)---the same as the experiences of the person you're talking to? We can't really answer the question of whether your qualitative experience of "red" is like my qualitative experience of "green" or, for that matter, "middle C." We still have no direct way to experience someone else's experiences. However, we can demonstrate that different people do, in some cases, inhabit different sensory worlds. Our discussion in this chapter will show how scaling methods can be used to perform this act of mind reading. FURTHER DISCUSSION of qualia can be found in Section 5.5. METHOD 3: SIGNAL DETECTION THEORY---MEASURING DIFFICULT DECISIONS A radiologist looks at a mammogram, the X-ray test used to screen for breast cancer. There's something on the X-ray that might be a sign of cancer, but it is not perfectly clear. What should the radiologist do? Suppose she decides to call it benign, not cancerous, and suppose she is wrong. Her patient might die. Suppose she decides to treat it as a sign of malignancy. Her patient will need more tests, perhaps involving surgery. The patient and her family will be terribly worried. If the radiologist is wrong and the spot on the mammogram is, in fact, benign, the consequences may be less dire than those of missing a cancer, but there will be consequences. This is a perceptual decision, made by an expert, that has real consequences. Our discussion of signal detection theory will show how decisions of this sort can be studied scientifically. METHOD 4: SENSORY NEUROSCIENCE Grilled peppers appear on your table as an appetizer. They have an appealing, smoky smell. When you bite into one, it has a complex flavor that includes some of that smokiness. Fairly quickly you also experience a burning sensation. There is no actual change in the temperature in your mouth, and your tongue is no warmer than it was, but the "burn" is unmistakable. How does the pepper fool your nervous system into thinking that your tongue is on fire? This chapter's exploration of sensory neuroscience will introduce the ways in which sensory receptors and nerves undergird your perceptual experience. METHOD 5: NEUROIMAGING---AN IMAGE OF THE MIND Suppose you arrange to view completely different images with different eyes. We might present a picture of a house to one eye and of a face to the other (Tong et al., 1998). The result would be an interesting effect known as binocular rivalry (see Section 6.4). The two images would compete to dominate your perception: sometimes you would see a house, and sometimes you would see a face. You would not see the two together. One reason binocular rivalry is interesting is that it represents a dissociation of the stimuli, presented to the eyes, and your private perceptual experience. Even if we cannot share the experience, modern brain-imaging techniques enable us to see traces of that experience as it takes place in the brain. Methods of neuroimaging will be our final methodological topic in this chapter. METHOD 6: COMPUTATIONAL MODELS If you're your first language is English, Spanish 'b' and 'p' might both sound like English 'b' to your ears, and French and German vowels are difficult to tell apart. Your ability to distinguish speech sounds depends almost entirely on the kinds of speech sounds that you heard while growing up. This is because, for all of your senses, perception is a combination of things one is born with and things that are acquired only through experience. Researchers might create a computational model to describe precisely how the basic abilities that every infant possesses at birth become shaped by experience in a particular language environment to predict the way you perceive speech sounds.1.2 Thresholds and the Dawn of Psychophysics Learning Objectives By the end of this section, you should be able to: 1.2.1Explain Weber's law, Fechner's law, and Steven's power law at a conceptual level. 1.2.2Describe commonly used psychophysical methods, including the method of constant stimuli, method of limits, method of adjustment, magnitude estimation, and cross-modality matching. 1.2.3Describe signal detection theory at a general level, including the concepts of sensitivity and criterion. Early on, study of the senses was a mix of experimental science and philosophy. Fascinating work can be found in ancient Greek philosophy, in medieval Islamic science, and in the writings of sages in China and India. We will start much later, with the very interesting and versatile nineteenth-century German scientist-philosopher Gustav Fechner (1801--1887) (FIGURE 1.3). FIGURE 1.3 Founder of experimental psychology Thought by some to be the true founder of experimental psychology, Gustav Fechner is best known for his pioneering work relating changes in the physical world to changes in our psychological experiences, thus inventing the field of psychophysics. View larger image Before making his first contributions to psychology, Fechner had an eventful personal history. Young Fechner earned his degree in medicine, but his interests turned from biological science to physics and mathematics. By 1833, he was a full professor of physics in Leipzig, Germany. Though this might seem an unlikely way to get to psychology, events proved otherwise. He became absorbed with the relationship between mind and matter. This pursuit placed him in the middle of a classic philosophical debate between adherents of dualism and materialism. Dualists hold that the mind has an existence separate from the material world of the body. Materialists hold that the mind is not separate. A modern materialist position, probably the majority view in scientific psychology, is that the mind is what the brain does. Fechner proposed to effectively split the difference by imagining that the mind, or consciousness, is present in all of nature. This panpsychism---the idea that the mind exists as a property of all matter---extended not only to animals, but to inanimate things as well. Fechner described his philosophy of panpsychism in a provocative book (first published, in German, in 1848) entitled Nanna, or Concerning the Mental Life of Plants. The title alone gives a pretty good idea of what Fechner had in mind. As a young scientist, Fechner overworked himself to exhaustion and severely damaged his eyes by gazing at the sun while performing vision experiments (a not-uncommon problem for curious vision researchers in the days before reliable, bright, artificial light sources). The visually incapacitated Fechner had some form of mental breakdown that left him sometimes unable to speak or eat. He resigned his university position, withdrew from almost all his friends and colleagues, and for 3 years spent almost all of his time alone with his thoughts. Fechner apparently solved his eating problem with a diet of "fruit, strongly spiced ham and wine" (Fancher, 1990, p. 133). His vision was also recovering. Then, while lying in bed on October 22, 1850 (a date still celebrated as "Fechner Day" by some), Fechner had a specific insight into the relationship between mental life and the physical world. From his experience as a physicist, Fechner thought it should be possible to describe the relation between mind and body using mathematics. His goal was to formally describe the relationship between sensation (mind) and the energy (matter) that gave rise to that sensation. He called both his methods and his theory psychophysics (psycho for "mind," and physics for "matter"). In his effort, Fechner was inspired by the findings of one of his German colleagues, Ernst Weber (1795--1878) (FIGURE 1.4), an anatomist and physiologist who was interested in touch. Weber tested the accuracy of our sense of touch by using a device much like the compass one might use to draw circles. He used this device to measure the smallest distance between two points that was required for a person to feel touch on two points of the skin instead of one. Later, Fechner would call this distance the two-point touch threshold. We will discuss two-point touch thresholds, and touch in general, in Chapter 13. FIGURE 1.4 Ernst Weber Weber discovered that the smallest detectable change in a stimulus, such as the weight of an object, is a constant proportion of the stimulus level. This relationship later became known as Weber's law. View larger image For Fechner, Weber's most important findings involved judgments of lifted weights. Weber would ask people to lift one standard weight (that is, a weight that stayed the same over a series of experimental trials) and one comparison weight that differed from the standard. Weber increased the comparison weight in incremental amounts over the series of trials. He found that the ability of a person to detect the difference between the standard and comparison weights depended greatly on the weight of the standard. When the standard was relatively light, people were much better at detecting a small difference when they lifted a comparison weight. When the standard was heavier, people needed a greater difference before they could detect a change. He called the difference required for detecting a change in weight the just noticeable difference, or JND. Another term for JND, the smallest change in a stimulus that can be detected, is the difference threshold. Weber noticed that JNDs change in a systematic way. The smallest change in weight that could be detected was always close to 1/40 of the standard weight. Thus, a 1-gram change could be detected when the standard weighed 40 grams, but a 10-gram change was required when the standard weighed 400 grams. Weber went on to test JNDs for a few other kinds of stimuli, such as the lengths of two lines (for which the detectable change ratio was 1:100). For virtually every measure---whether brightness, pitch, or time---a constant ratio between the change and the standard could describe the threshold of detectable change quite well. This ratio rule holds true except for extreme stimuli---stimuli so small or large that they approach the minimum or maximum of our senses. In recognition of Weber's discovery, Fechner called these ratios, or proportions, Weber fractions, and he called the mathematical formula that described the general rule Weber's law. Weber's law states that the size of the just detectable difference (ΔI) is a constant proportion (K) of the level of the stimulus (I). In Weber's observations, Fechner found what he was looking for: a way to describe the relationship between mind and matter. Fechner assumed that the smallest detectable change in a stimulus (ΔI) could be considered a unit of the mind because this is the smallest bit of change that is perceived. He then mathematically extended Weber's law to create what became known as Fechner's law (FIGURE 1.5): 𝑆 = 𝑘 log 𝑅 where S is the psychological sensation, which is equal to the logarithm of the physical stimulus level (log R) multiplied by a constant, k. This equation describes the fact that our psychological experience of the intensity of light, sound, smell, taste, or touch increases less quickly than the actual physical stimulus increases. With this equation, Fechner provided us with at least one way to relate mind and matter. FIGURE 1.5 Fechner's law As the intensity of a physical stimulus increases (x-axis), a larger change in that physical stimulus is required to produce a just detectable difference in sensation (y-axis). This is seen graphically in the increasing amount of space between the dashed lines on the x-axis is required to produce evenly spaced dashed lines on the y-axis. The relationship between X and Y values is logarithmic. View larger image Even if mind and matter are related, we take care to distinguish between units of physical entities (such as light or sound) and measures of people's perception ("brightness," "loudness"). For example, the physical intensity of a sound---the sound pressure level---is a physical entity we can measure in decibels, whereas a person's perception of "loudness" is psychophysical and subjective (see Section 9.2). Similarly, frequency is a measure of the rate of fluctuations of the physical sound pressure, while the "pitch" of a musical note describes a psychophysical response to that physical phenomenon. Frequency and pitch are not the same thing, although they are closely correlated. Over a wide range, as frequency increases, so does pitch, though it is unclear whether there is a perception of pitch for the highest audible frequencies (Green, 2005). Fechner was the first to objectively measure psychological events through new ways to measure what people see, hear, and feel (Wixted, 2020). As such he can be considered to be the true founder of experimental psychology (Boring, 1950), even if that title is usually given to Wilhelm Wundt (1832--1920), who began his work sometime later. All of Fechner's methods are still in use today. In explaining these methods here, we will use absolute threshold as an example because it is perhaps the most straightforward; but we would use the same methods to determine difference thresholds such as ΔI. An absolute threshold is the minimum intensity of a stimulus that can be detected (TABLE 1.1). This returns us to the question we raised earlier: What is the faintest sound you can hear? Of course, we can ask the same question about the faintest light, the lightest touch, and so forth. (See Activity 1.1: Psychophysics.) TABLE 1.1 Absolute thresholds in the real world Sense Threshold Vision Stars at night, or a candle flame 30 miles away on a dark, clear night Hearing A ticking watch 20 feet away, with no other noises Vestibular A tilt of less than half a minute on a clock face Taste A teaspoon of sugar in 2 gallons of water Smell A drop of perfume in 3 rooms Touch The wing of a fly falling on your cheek from a height of 3 inches Source: From E. Galanter. 1962. In New directions in psychology. T. Newcomb et al. (Eds.), Holt, Rinehart and Winston: New York. Psychophysical Methods How can we measure an absolute threshold in a valid and reliable manner? One method, known as the method of constant stimuli, requires creating many stimuli with different intensities in order to find the tiniest intensity that can be detected (FIGURE 1.6). If you've had a hearing test, you had to report when you could and could not hear a tone that the audiologist played to you over headphones, usually in a very quiet room. In this test, intensities of all of the tones were relatively low, not too far above or below the intensity where your threshold was expected to be. The tones, varying in intensity, were presented randomly, and tones were presented multiple times at each intensity. FIGURE 1.6 The method of constant stimuli (A) We might expect the threshold to be a sharp change in detection from never reported to always reported, as depicted here, but this is not so. (B) In reality, experiments measuring absolute threshold produce shallower functions relating stimulus to response. A somewhat arbitrary point on the curve, often 50% detection, is designated as the threshold (dashed line). View larger image The "multiple times" piece is important. Subtle perceptual judgments such as threshold judgments are variable. The stimulus varies for physical reasons. The observer varies. Attention waivers and sensory systems fluctuate for all sorts of reasons. As a consequence, one measure is almost never enough. You need to repeat the measure over and over and then average the responses or otherwise describe the pattern of results. Some experiments require thousands of repetitions (thousands of "trials") to establish a sufficiently reliable data point. Returning to our auditory example, as the listener, you would report whether you heard a tone or not. You would always report hearing a tone that was relatively far above threshold, and almost never report hearing a tone that was well below threshold. In between, however, you would be more likely to hear some tone intensities than not to hear them, and you would hear other, lower intensities on only a few presentations. In general, the intensity at which a stimulus would be detected 50% of the time would be chosen as your threshold. That 50% definition of absolute threshold is rather interesting. Weren't we looking for a way to measure the weakest detectable stimulus? Using the hearing example, shouldn't that be a value below which we just can't hear anything (see Figure 1.6A)? It turns out that no such hard boundary exists. Because of variability in the nervous system, stimuli near threshold will be detected sometimes and missed at other times. As a result, the function relating the probability of detection with the stimulus level will be gradual (see Figure 1.6B), and we must settle for a somewhat arbitrary definition of an absolute threshold. We will return to this issue when we talk about signal detection theory. The method of constant stimuli is simple to use, but it is an inefficient way to conduct an experiment, because much of the listener's time is spent with stimuli that are well above or below threshold. A more efficient approach is the method of limits (FIGURE 1.7). With this method, the experimenter begins with the same set of stimuli---in this case, tones that vary in intensity. Instead of random presentations, tones are presented in order of increasing or decreasing intensity. When tones are presented in ascending order, from faintest to loudest, listeners are asked to report when they first hear the tone. With descending order, the task is to report when the tone is no longer audible. Data from an experiment such as this show that there is some "overshoot" in judgments. It usually takes more intensity to report hearing the tone when intensity is increasing, and it takes more decreases in intensity before a listener reports that the tone cannot be heard. We take the average of these crossover points---when listeners shift from reporting hearing the tone to not hearing the tone, and vice versa---to be the threshold. FIGURE 1.7 The method of limits Here the listener attends to multiple series of trials. For each series, the intensity of the stimulus is gradually increased or decreased until the listener detects (Y) or fails to detect (N), respectively, the stimulus. For each series, an estimate of the threshold (red dashed line) is taken to be the average of the stimulus level just before and after the change in perception (i.e., the average "crossover value"). View larger image The third and final of these classic measures of thresholds is the method of adjustment. This method is just like the method of limits, except the person being tested is the one who steadily increases or decreases the intensity of the stimulus. The method of adjustment may be the easiest method to understand, because it is much like day-to-day activities such as adjusting the volume dial on a stereo or the dimmer switch for a light. Even though it's the easiest to understand, the method of adjustment is not usually used to measure thresholds. The method would be perfect if threshold data were like those plotted in Figure 1.6A, but a graph of real data looks more like Figure 1.6B. The same person will adjust a dial to different places on different trials, and measurements get even messier when we try to combine the data from multiple people. Scaling Methods Moving beyond absolute thresholds and difference thresholds, suppose we wanted to know how strong your experiences are. For example, we might show you a light and ask how much additional light you would need to make another light look twice as bright. Though that might seem like an odd question, it turns out to be answerable. We could give you a knob to adjust so that you could set the second light to appear twice as bright as the first, and you could do it. In fact, we don't need to give the observers a light to adjust. A surprisingly straightforward way to address the question of the strength or size of a sensation is to simply ask observers to rate the experience. For example, we could give observers a series of sugar solutions and ask them to assign numbers to each sample. We would just tell our observers that sweeter solutions should get bigger numbers, and if solution A seems twice as sweet as solution B, the number assigned to A should be twice the number assigned to B. This method is called magnitude estimation, and the approach actually works well, even when observers are free to choose their own range of numbers. More typically, however, we might begin the experiment by presenting one solution at an intermediate level and telling the taster to label this level as a specific value---10, for instance. All of the responses should then be scaled sensibly above or below this standard of 10. If you do this for sugar solutions, you will get data that look like the blue "sweetness" line in FIGURE 1.8. FIGURE 1.8 Magnitude estimation The lines on this graph represent data from magnitude estimation experiments using electric shocks of different currents, lines of different lengths, solutions of different sweetness levels, and lights of different brightness levels. The exponents for the "power functions" that describe these lines are 3.5, 1.0, 0.8, and 0.3, respectively. For exponents greater than 1, such as for electric shock, Fechner's law does not hold, and Stevens's power law must be used instead. View larger image Inspired by his student Richard Held, a distinguished vision researcher whose work you will learn about in Sections 6.4 and 13.1, Harvard psychologist S. S. Stevens (1962, 1975) developed magnitude estimation. Stevens, his students, and their successors measured functions like the one in Figure 1.8 for many different sensations. Even though observers were asked to assign numbers to private experience, the results were orderly and lawful. However, they were not the same for every type of sensation. That relationship between stimulus intensity and sensation is described by what is now known as Stevens's power law: 𝑆 = 𝑎 𝐼 𝑏 which states that the sensation (S) is related to the stimulus intensity (I) by an exponent (b). (The letter a is a constant that corrects for the units you are using. For example, if you measured your stimulus in meters and then switched to measuring it in centimeters, you would need to multiply by 0.01 \[= divide by 100\] to keep your sensation numbers the same.) So, for example, experienced sensation might rise with intensity squared (I × I). That would be an exponent of 2.0. If the exponent is less than 1, it means that the sensation grows less rapidly than the stimulus---which is what Fechner's law and Weber's law would predict. Suppose you have some lit candles and you light 10 more. If you start with 1 candle, the change from 1 to 11 candles must be quite dramatic. If you start with 100 and add 10, the change will be modest. Adding 10 to 10,000 won't even be noticeable. In fact, the exponent for brightness is about 0.3. The exponent for sweetness is about 0.8 (Bartoshuk, 1979). Properties like length have exponents near 1, so, reasonably enough, a 12-inch-long stick looks twice as long as a 6-inch-long stick (S. S. Stevens and Galanter, 1957). Note that this length relationship is true over only a moderate range of sizes. An inch added to the size of a spider changes your sensory experience much more than an inch added to the height of a giraffe. Some stimuli have exponents greater than 1. In the painful case of electric shock, the pain grows with I3.5 (Stevens, Carton, and Shickman, 1958), so a 4-fold increase in the electrical current is experienced as a 128-fold increase in pain! Weber's and Fechner's laws have rather broad implications beyond questions of apparent brightness or loudness. Some plants, for example, will respond to the psychophysical experience of the bees that pollinate them (Nachev et al., 2017). Suppose you are a plant pollinated by bees that are attracted to your flowers because those flowers have sweet nectar. How much sugar do you need to put into that nectar? After all, it's going to cost you energy to produce sugar. If your nectar has 2 units of sugar and the neighbor flower only has 1, you probably have a competitive advantage over the neighbor. However, if yours has 12 units and the neighbor's has 11, that difference of 1 unit might fall below the bee's Weber fraction for sweetness. Thus, there will be greater evolutionary pressure to go from 1 to 2 than from 11 to 12. Similarly, a peacock with 51 feathers in his tail probably does not have much of a reproductive advantage over a 50-feather peacock. The peahen might not notice the difference, putting an evolutionary brake on tail inflation (Farris, 2017). At this point in our discussion of psychophysics, it is worth taking a moment to compare Weber's, Fechner's, and Stevens's laws: Weber's law involves a clear objective measurement. We know how much we varied the stimulus, and either the observers can tell that the stimulus changed or they cannot. Fechner's law begins with the same sort of objective measurements as Weber's, but the law is actually a calculation based on some assumptions about how sensation works. In particular, Fechner's law assumes that all JNDs are perceptually equivalent. In fact, this assumption turns out sometimes to be incorrect and leads to instances where the "law" is violated, such as in the electric shock example just given. Stevens's power law describes rating data quite well, but notice that rating data are qualitatively different from the data that support Weber's law. We can record the observer's ratings and we can check whether those ratings are reasonable and consistent, but there is no way to know whether they are objectively right or wrong. A useful variant of the scaling method shows us that different individuals can live in different sensory worlds, even if they are exposed to the same stimuli. This method is called cross-modality matching. In cross modality matching, an observer adjusts a stimulus of one sort to match the perceived magnitude of a stimulus of a completely different sort (J. C. Stevens, 1959). For example, we might ask a listener to adjust the brightness of a light until it matches the loudness of a particular tone. Again, though the task might sound odd, people can do this, and for the most part, everyone with "normal" vision and hearing will produce a similar pattern of matches of a sound to a light. We still can't examine someone else's private experience, but at least the relationship of visual experience and auditory experience appears to be similar across individuals. This similarity does not hold when it comes to the sense of taste. There is a molecule called propylthiouracil (PROP) that some people experience as very bitter, while others experience it as almost tasteless. Still others fall in between. This relationship between a chemical and bitter taste can be examined formally with cross-modality matching (Marks et al., 1988). When observers are asked to match the bitterness of PROP to other sensations completely unrelated to taste, we do not find the sort of agreement that is found when observers match sounds and lights (FIGURE 1.9). Some people---we'll call them nontasters---match the taste of PROP to very weak sensations, like the sound of a watch or a whisper. A group of "supertasters" assert that the bitterness of PROP is similar in intensity to the brightness of the sun or the most intense pain ever experienced. Medium tasters match PROP to weaker stimuli, such as the smell of frying bacon or the pain of a mild headache (Bartoshuk, Fast, and Snyder, 2005). As we will see in Section 15.5, there is a genetic basis for this variation, and it has wide implications for our food preferences and, consequently, for health. For the present discussion, this example shows that we can use scaling methods to quantify what appear to be real differences in individuals' taste experiences. FIGURE 1.9 Cross-modality matching The levels of bitterness of concentrated PROP perceived by nontasters, medium tasters, and supertasters of PROP are shown on the left. The perceived intensities of a variety of everyday sensations are shown on the right. The arrow from each taster type indicates the level of sensation to which those tasters matched the taste of PROP. (Data from K. Fast. 2004. Developing a Scale to Measure Just About Anything: Comparisons across Groups and Individuals. Thesis Digital Library, 3353. Yale University School of Medicine: New Haven, CT.) View larger image Signal Detection Theory Let's return to thresholds---particularly to the fact that they are not absolute. An important way to think about this fact and to deal with it experimentally is known as signal detection theory (D. M. Green and Swets, 1966). Like so much of modern psychophysics, even signal detection theory was anticipated by Fechner over a hundred years earlier (Wixted, 2020). Signal detection theory begins with the fact that the stimulus you're trying to detect (the "signal") is always being detected in the presence of "noise." If you sit in the quietest place you can find and put on your best noise-canceling headphones, you will find that you can still hear something. Similarly, if you close your eyes in a dark room, you still see something---a mottled pattern of gray with occasional brighter flashes. This is internal noise, the static in your nervous system. Many neurons in the brain are firing all the time, even when nothing is happening. For example, many neurons in the auditory system fire up to 50 times per second when there is no sound at all, and you will learn in Chapter 12 that neurons in the vestibular system fire 100 times per second even when you're perfectly motionless. When you're trying to detect a faint sound or flash of light, you must be able to detect it in the presence of such internal noise. Near your threshold, it will be hard to tell a real stimulus from a random surge of internal noise. There is external noise, too. Consider again that radiologist reading a mammogram looking for signs of breast cancer. As you can see in FIGURE 1.10, the mammogram contains lots of similar regions; the marked white region is the danger sign. We can think of the cancer as the signal. By the time it is presented to the radiologist in an X-ray, there is a signal plus noise. Elsewhere in the image, and in other images, are stimuli that are nothing other than noise. The radiologist is a visual expert, trained to find these particular signals, but sometimes the signal will be lost in the noise and missed, and sometimes some noise will look enough like cancer to generate a false alarm (Nodine et al., 2002). Thus, the radiologist will be faced with uncertainty, introduced both as external and internal noise. FIGURE 1.10 Differentiating signal from noise Mammograms---X-rays of the breast---are used to screen women for breast cancer. The solid white region is the signal of a cancerous growth; however, the mammogram contains many similar regions ("noise"). Reading such images is a difficult perceptual task. Even for a radiologist trained to discriminate and identify particular signals, there is always uncertainty due to internal and external noise. © Xray Computer/Shutterstock.com View larger image Of course, sometimes neither internal nor external noise is much of a problem. When you see this dot, , you are seeing it in the presence of internal noise, but the magnitude of that noise is so much smaller than the signal generated by the dot that the noise has no real impact. Similarly, the dot may not be exactly the same as other dots, but that variation---the external noise---is also too small to have an impact. If asked about the presence of a dot here, , and its absence here, , you will be correct in your answer essentially every time. Signal detection theory exists to help us understand what's going on when we make decisions under conditions of uncertainty. Because we are not expert mammographers, let's introduce a different example to illustrate the workings of signal detection theory. You're in the shower. The water is making a noise that we will imaginatively call "noise." Sometimes the noise sounds louder to you; sometimes it seems softer. We can plot the distribution of your perception of noise as shown in FIGURE 1.11A. On the x-axis, we have the magnitude of your sensation from "less" to "more." Imagine that we asked you, over and over again, about your sensation. Or imagine we took many repeated measures of the response in your nervous system to the sound. For some instances, the response would be "less." For some, it would be "more." On average, it would lie somewhere in between. If we tabulated all of the responses, we would get a bell-shaped (or "normal") distribution of answers, with the peak of that distribution showing the average answer that you gave. Now a ringtone plays. That will be our "signal." Your perceptual task is to detect the signal in the presence of the noise. What you hear is a combination of the ringtone and the shower. That is, the signal is added to the noise, so we can imagine that now we have two distributions of responses in your nervous system: a noise-alone distribution and a signal-plus-noise distribution (FIGURE 1.11B). For the sake of simplicity, let's suppose that "more" response means that it sounds more like the phone is ringing. So now your job is to decide whether it's time to jump out of the shower and answer what might be the phone. The problem is that you have no way of knowing at any given moment whether you're hearing noise alone or signal plus noise. The best you can do is to decide on a criterion level of response (FIGURE 1.11C). If the response in your nervous system exceeds that criterion, you will jump out of the shower and run naked and dripping to find the phone. If the level is below the criterion, you will decide that it is not a ringtone and stay in the shower. Note that this "decision" is made automatically; it's not that you sit down to make a conscious (soggy) choice. Thus, in signal detection theory, a criterion is a value that is somehow determined by the observer. Within the observer, a response above the criterion will be taken as evidence that a signal is present. A response below that level will be treated as noise. There are four possible outcomes in this situation: You might say "no" when there is no ringtone; that's a correct rejection or true negative (FIGURE 1.11D). You might say "yes" when there is a ringtone; that's known as a hit or true positive response (FIGURE 1.11E). Then there are the errors. If you jump out of the shower when there's no ringtone, that's a false alarm or false positive (FIGURE 1.11F). If you miss the call, that's a miss or false negative (FIGURE 1.11G). FIGURE 1.11 Detecting a stimulus using signal detection theory (SDT) (A) SDT assumes that all perceptual decisions are made against a background of noise (the red curve) generated both in the world and in the nervous system. (B) Your job is to distinguish nervous system responses due to noise alone (dotted, red curve) or to signal plus noise (solid, blue curve). (C) The best you can do is establish a criterion (solid black line) and declare that you detect something if the response is above that criterion. (D--G) Signal detection theory includes four classes of responses. (D) "Correct rejection" (you say "no" and there is, indeed, no signal). (E) "Hits" (you say "yes" and there is a signal). (F) "False-alarm errors" (you say "yes" to no signal). (G) "Miss errors" (you say "no" to a real signal). View larger image How sensitive are you to the ringtone? In FIGURE 1.12, the sensitivity is graphed as the separation between the noise-alone and signal-plus-noise distributions. If the distributions are on top of each other (Figure 1.12A), you can't tell noise alone from signal plus noise. A false alarm is just as likely as a hit. By knowing the relationship of hits to false alarms, you can calculate a sensitivity measure known as d ʹ (d-prime), which would be about zero in Figure 1.12A. In Figure 1.12C we see the case of a large d ʹ. Here you could detect essentially all the ringtones and never make a false alarm error. The situation we've been discussing is in between (Figure 1.12B). FIGURE 1.12 Sensitivity (d ʹ) in SDT Your sensitivity to a stimulus is illustrated by the separation between the distributions of your response to noise alone (dotted, red curve) and to signal plus noise (solid, blue curve). This separation is captured by the measure d ʹ (d-prime). (A) If the distributions completely overlap, d ʹ is almost 0 and you have no ability to detect the signal. (B) If d ʹ is intermediate, you have some sensitivity but your performance will be imperfect. (C) If d ʹ is big, then distinguishing signal from noise is easy. View larger image Now suppose you're waiting for an important call. Even though you really don't want to miss the call, you can't magically make yourself more sensitive. All you can do is move the criterion level of response, as shown in FIGURE 1.13. If you shift your criterion to the left, you won't miss many calls, but you will have lots of false alarms (Figure 1.13A). That's annoying. You're running around naked, dripping on the floor, and traumatizing the cat for no good reason. If you shift your criterion to the right, you won't have those annoying false alarms, but you will miss most of the calls (Figure 1.13C). For a fixed value of d ʹ, changing the criterion changes the hits and false alarms in predictable ways. If you plot false alarms on the x-axis of a graph against hits on the y-axis for different criterion values, you get a curve known as a receiver operating characteristic (ROC) curve (FIGURE 1.14). FIGURE 1.13 Criterion in SDT For a fixed d ʹ, all you can do is change the pattern of your errors by shifting the response criterion. If you don't want to miss any signals, you move your criterion to the left (A), but then you have more false alarms. If you don't like false alarms, you move the response criterion to the right (C), but then you make more miss errors. In all these cases (A--C), your sensitivity, d ʹ, remains the same. View larger image FIGURE 1.14 Receiver operating characteristic (ROC) curves Theoretical ROC curves for different values of d ʹ. Note that d ʹ = 0 when performance is at the chance level. Higher values of d ʹ indicate that the probability of hits and correct rejections increases, and the probability of misses and false alarms decreases. Pr(N\|n) = probability of the response "no signal present" when no signal is present (correct rejection); Pr(N\|s) = probability of the response "no signal present" when signal is present (miss); Pr(S\|n) = probability of the response "signal present" when no signal is present (false alarm); Pr(S\|s) = probability of the response "signal present" when signal is present (hit). View larger image Suppose you were guessing Figure 1.12A situation). You might guess "yes" on 40% of the occasions when the phone rang, but you would also guess "yes" on 40% of the occasions when the phone did not ring. If you moved your criterion and guessed "yes" on 80% of phone-present occasions, you would also guess "yes" on 80% of phone-absent occasions. Your data would fall on that "chance performance" diagonal in Figure 1.14. If you were perfect (the Figure 1.12C situation), you would have 100% hits and 0% false alarms and your data point would lie at the upper left corner in Figure 1.14. Situations in between (Figure 1.12B) produce curves between guessing and perfection (the green, purple, and blue curves in Figure 1.14). If your data lie below the chance line, you did the experiment wrong! Let's return to our radiologist. She will have an ROC curve whose closeness to perfection reflects her expertise. On that ROC, her criterion can slide up and to the right, in which case she will make more hits but also more false alarms, or down and to the left, in which case she will have fewer false alarms but more misses. Where she places her criterion (consciously or unconsciously) will depend on many factors. Does the patient have factors that make her more or less likely to have cancer? What is the perceived cost of a missed cancer? What is the perceived cost of a false alarm? You can see that what started out as a query about the lack of absolute thresholds can become, quite literally, a matter of life and death. Signal detection theory can become a rather complicated topic in detail (e.g., what happens if those noise and signal + noise curves are not exactly the same shape?). To learn about how to calculate d ʹ and about ROC curves, you can take advantage of many useful websites and several texts (e.g., Macmillan and Creelman, 2005; see Burgess, 2010 if you're interested in the application to radiology). 1.3 Sensory Neuroscience and the Biology of Perception Learning Objectives By the end of this section, you should be able to: 1.3.1Explain the doctrine of specific nerve energies. 1.3.2List the lobes of the brain and what senses are processed in each one. 1.3.3Describe the basic anatomy of neurons and how they transmit information. 1.3.4Describe the different neuroimaging techniques covered in the chapter: EEG, ERP, MEG, MRI, fMRI, and PET. Many of you reading this book will have had some introduction to neuroscience. Here, we review very briefly some of the neuroscience that is relevant to the study of sensation and perception. If this is your first encounter with neuroscience, you may want to consult a neuroscience text to give yourself a more detailed background than we will provide. During the nineteenth century, when Weber and Fechner were initiating the experimental study of perception, physiologists were hard at work learning how the senses and the brain operate. Much of this work involved research on animals. It's worth spending a moment on a key assumption here: studies of animal senses tell us something about human senses. That may seem obvious, but the assumption requires the belief that there is some continuity between the way animals work and the way humans work. The most powerful argument for a continuity between humans and animals came from Darwin's theory of evolution. During the 1800s, Charles Darwin (1809--1882) proposed his revolutionary theory in On the Origin of Species (1859). Although many of the ideas found in that book had been brewing for some time, controversy expanded with vigor following Darwin's provocative statements in The Descent of Man (1871), where he argued that humans and apes evolved from a common ancestor. If there was continuity in the structure of the bones, heart, and kidneys of cows, dogs, monkeys, and humans, then why wouldn't there be continuity in the structure and function of their sensory and nervous systems? An inescapable implication of the theory of evolution is that we can learn much about human sensation and perception by studying the structure and function of our nonhuman relatives. Nerves and Specific Nerve Energies At the same time that Darwin was at work in England, the German physiologist Johannes Müller (1801--1858) (FIGURE 1.15) was writing his very influential Handbook of Physiology during the early 1830s. In this book, in addition to covering most of what was then known about physiology, Müller formulated the doctrine of specific nerve energies. The central idea of this doctrine is that we cannot be directly aware of the world itself, and we are only aware of the activity in our nerves. Further, what is most important is which nerves are stimulated, and not how they are stimulated. For example, we experience vision because the optic nerve leading from the eye to the brain is stimulated, but it does not matter whether light, or something else, stimulates the nerve. To prove to yourself that this is true, close your eyes and press very gently on the outside corner of one eye through the lid. (This works better in a darkened room.) You will see a spot of light toward the inside of your visual field by your nose. Despite the lack of stimulation by light, your brain interprets the input from your optic nerve as informing you about something visual. FIGURE 1.15 Johannes Müller The German physiologist Johannes Müller formulated the doctrine of specific nerve energies, which says that we are aware only of the activity in our nerves, and we cannot be aware of the world itself. For this reason, what is most important is which nerves are stimulated, not how they are stimulated. View larger image The cranial nerves leading into and out of the skull illustrate the doctrine of specific nerve energies (FIGURE 1.16). The pair of optic nerves is one of 12 pairs of cranial nerves that pass through small openings in the bone at the base of the skull; these nerves are dedicated mainly to sensory and motor systems. Cranial nerves are labeled both by names (in boldface type) and by Roman numerals that roughly correspond to the order of their locations, beginning from the front of the skull. Three of them---olfactory (I), optic (II), and vestibulocochlear (VIII)---are exclusively dedicated to sensory information. The vestibulocochlear nerve serves two sensory modalities: the vestibular sensations that support our sense of equilibrium (see Chapter 12) and hearing (discussed in Section 9.3). Three more cranial nerves---oculomotor (III), trochlear (IV), and abducens (VI)---are dedicated to muscles that move the eyes. In addition to instructions from the visual system, these nerves have direct connections from the vestibular system so your eyes know where your nose is pointed. The other six cranial nerves either are exclusively motor (spinal accessory \[XI\] and hypoglossal \[XII\]), or convey both sensory and motor signals (trigeminal \[V\], facial \[VII\], glossopharyngeal \[IX\], and vagus \[X\]). FIGURE 1.16 Cranial nerves Twelve pairs of cranial nerves pass through small openings in the bone at the base of the skull. These nerves conduct information for sensation, motor behavior, or both. (After S. M. Breedlove and N. V. Watson. 2013. Biological Psychology: An Introduction to Behavioral, Cognitive, and Clinical Neuroscience, 7th ed., Oxford University Press/Sinauer Associates: Sunderland, MA.) View larger image The doctrine of specific nerve energies extends beyond the cranial nerves, as illustrated especially well by our senses of hot and cold on the skin. Two specialized types of nerve cells are warmth fibers and cold fibers, which respond to increases and decreases in temperature on the skin (see Section 13.1). Capsaicin, a chemical that occurs naturally in chili peppers, causes warmth fibers to fire, creating a sense of increasing heat even though the temperature has not changed. On the other hand, menthol, which imparts a minty flavor in cough drops, stimulates cold fibers (Bautista et al., 2007), so skin feels cooler without getting physically colder. In sufficiently high amounts, both capsaicin and menthol stimulate pain receptors in the skin. Paradoxically, this is why ointments often contain capsaicin or menthol, where their effect is to mask real physical pain (Green, 2005). Just as different nerves are dedicated to individual sensory and motor tasks, areas of the brainstem and cerebral cortex are similarly dedicated to particular tasks. Areas of the cortex dedicated to perception actually are much larger than the darkened areas in FIGURE 1.17. The areas depicted here are primary sensory areas; more complex processing is accomplished across cortical regions that spread well beyond these primary areas. For example, visual perception uses cortex that extends both anteriorly (forward) into parietal cortex and ventrally (lower) into regions of the temporal lobe (see Figures 4.2 and 4.4). In addition, as processing extends beyond primary areas, cortex often becomes polysensory, meaning that information from more than one sense is being combined in some manner. (See Activity 1.2: Sensory Areas in the Brain.) While this textbook is organized into sense-specific chapters, we always want to remember that we live in a multisensory world. Researchers specifically study what is called sensory integration or multisensory integration (Spence, 2018). Throughout this book, you will find multisensory research, flagged with. FIGURE 1.17 Cortex of the human brain The cortex is the outer layer of the cerebral hemispheres. This lateral view shows the left hemisphere. The anterior (front of head) is on the left and the posterior (back of the head) is on the right. The darkened areas show where information from four of our sensory modalities first reaches the cortex. Also shown is the motor cortex, which is engaged in balance, touch, and some auditory processing. View larger image Hermann Ludwig Ferdinand von Helmholtz (1821--1894) (FIGURE 1.18), one of the greatest scientists of the nineteenth century, was greatly influenced by Müller at the Free University of Berlin. In an early effort, Helmholtz estimated that the speed of signal transmission in the nerves in frog legs was about 90 feet per second, comfortably within the range of ordinary physical events. Later he concluded that human sensory nerves transmitted signals at speeds of between 165 and 330 feet per second. (See Activity 1.3: Neurons.) This transmission speed was slower than many people believed at the time, probably because it feels like you are sensing what is happening right now. In fact, you are experiencing recent (very recent) history. To use Helmholtz's example, a "whale probably feels a wound near its tail in about one second and requires another second to send back orders to the tail to defend itself" (Koenigsberger, 1906/1965, p. 72). Some neurons are faster than others but, regardless, it is interesting to realize that when you stub your toe, a measurable amount of time elapses before you feel the consequences. FIGURE 1.18 Hermann von Helmholtz Contributor to many fields of science, von Helmholtz was one of the greatest scientists of all time. He made many important discoveries in physiology and perception. View larger image Neuronal Connections The neuron that carries a signal from your toe to your brain needs to pass its information on to other neurons. How do neurons connect with each other? The Spanish neuroanatomist Santiago Ramón y Cajal (1852--1934) (FIGURE 1.19A) looked closely at thin slices of tissue and created some of the most painstaking and breathtaking drawings of the organization of neurons in the brain. FIGURE 1.19B shows an example. His drawings suggested that neurons do not actually touch one another. Instead, he depicted neurons as separate cells with tiny gaps between them. Sir Charles Scott Sherrington (1857--1952) (FIGURE 1.20) named the tiny gap between the axon of one neuron and the dendrite of the next a synapse, from the Greek word meaning "to fasten together" (FIGURES 1.21 and 1.22). FIGURE 1.19 Santiago Ramón y Cajal (A) Santiago Ramón y Cajal (looking tired; perhaps this was not his happiest day in the lab) created meticulous drawings of brain neurons (B) while peering into a microscope for many hours. Because of his painstaking care and accuracy, his early drawings are still cited and reproduced today. View larger image FIGURE 1.20 Sir Charles Scott Sherrington An English neurophysiologist, Sherrington earned a Nobel Prize for his pioneering discoveries concerning neural activities. Neuroscience History Archives, Brain Research Institute, University of California, Los Angeles View larger image FIGURE 1.21 A typical neuron Features of a typical neuron and their functions are described. Axon length and dendrite branching patterns vary greatly across the different types of neurons. View larger image FIGURE 1.22 A synapse An axon terminal in the presynaptic cell communicates with a dendrite of the postsynaptic cell. Neurotransmitter molecules are released by synaptic vesicles in the axon and fit into receptors on the dendrite on the other side of the synapse, thus communicating from the axon of the first (presynaptic) neuron to the dendrite of the second (postsynaptic) neuron. View larger image Initially, people thought that some sort of electrical wave traveled across the synapse from one neuron to the next. However, Otto Loewi (1873--1961) (FIGURE 1.23) was convinced that this could not be true. One reason is that some neurons increase the response of the next neuron (they are excitatory), whereas other neurons decrease the response of the next neuron (they are inhibitory). Loewi proposed that something chemical, instead of electrical, might be at work at the synapse. The chemicals turned out to be molecules, called neurotransmitters, that travel from the axon across the synapse to bind to receptor molecules on the dendrite of the next neuron. There are many different kinds of neurotransmitters in the brain, and individual neurons are selective with respect to which neurotransmitters excite or inhibit them from firing. Drugs that are psychoactive, such as amphetamines, work by increasing or decreasing the effectiveness of different neurotransmitters. Today, scientists use chemicals that influence the effects of neurotransmitters in efforts to understand pathways in the brain, including those used in perception. FIGURE 1.23 Otto Loewi The German pharmacologist, Otto Loewi, received a Nobel Prize for demonstrating that neurons communicate with one another by releasing chemicals called neurotransmitters. Wellcome Collection/CC BY 4.0 View larger image Neuronal Firing: The Action Potential After Loewi's discovery of neurotransmitters, scientists learned what it really means to have a neuron "fire." Important early research took advantage of the fact that giant neurons of some squids have axons as thick as 1 millimeter. At their laboratory in the seaport of Plymouth, England, Sir Alan Hodgkin (1914--1998) and Sir Andrew Huxley (1917--2012) (FIGURE 1.24) would study squid fresh off the fishing boat. They would isolate a single giant neuron from the squid and test how the nerve impulse traveled along the axon. With such large axons, they could pierce the axon with an electrode to measure voltage, and could even inject different chemicals inside. They learned that neuronal firing is actually electrochemical (FIGURE 1.25). Voltage increases along the axon are caused by changes in the membrane of the neuron that permit positively charged sodium ions (Na+) to rush very quickly into the axon from outside the cell. Then, very quickly, the membrane changes again in a way that pushes positively charged potassium ions (K+) out of the axon, restoring the neuron to its initial resting voltage. All of this---sodium in and potassium out---occurs in about a thousandth of a second every time a neuron fires. You can read about the details in their Nobel Prize lectures (Hodgkin, 1964; Huxley, 1964). FIGURE 1.24 First recordings from inside a neuron British physiologists, Sir Alan Hodgkin (A) and Sir Andrew Huxley (B), earned a Nobel Prize for discovering how the action potential works to conduct signals along the axons of neurons. They moved from Cambridge to the coastal city of Plymouth, England, to record voltages from inside the giant axons of freshly caught Atlantic squid (C). A © Keystone Press/Alamy Stock Photo; B © PA Images/Alamy Stock Photo View larger image FIGURE 1.25 Generating an action potential A neuron "fires" when a stimulus makes the voltage across a piece of the cell membrane a bit more positive than its negative "resting potential". This is called depolarization. Depolarization permits sodium ions (Na+) to rush into the cell, thus increasing the voltage and generating an action potential. Very quickly afterward, potassium ion (K+) flow out of the cell, bringing the voltage back to the resting voltage. An action potential is analogous to a wave, sweeping across the ocean or around a football stadium. Here, the action potential sweeps along the length of the axon until it reaches the axon terminal. View larger image Because even the largest axons in mammals are much, much smaller than the giant squid axon, it is difficult to insert an electrode inside a neuron. Usually we measure electrical changes from just outside mammalian neurons (FIGURE 1.26). By measuring different aspects of neurons firing, we can learn about how individual neurons encode and transmit information from sense organs through higher levels of the brain. FIGURE 1.26 Listening to neurons Neuroscientists record the activity of single neurons with electrodes placed close to the axons. Modern techniques permit the insertion of multiple electrodes to measure activity in multiple neurons so that we can better understand how neurons work in concert when encoding sensory information. View larger image One way to investigate what a neuron encodes is to try to identify the stimulus that makes it fire the most vigorously. For example, a neuron in the primary visual cortex (see Section 3.4) might respond best to lines that are vertical, less to lines tilted to the left or right, and not at all to horizontal lines. When you read about the auditory system in Chapter 9, you will see that the timing and the rate of firing are important. Sometimes we learn a lot by simply finding the threshold that gets a neuron to fire at all. For example, you will learn that different neurons in the auditory nerve, on the way from the ear to the brain, are most sensitive to particular frequencies of sounds. In FIGURE 1.27, you can see curves obtained by measuring responses of six different neurons to sounds of different frequencies and intensities. These are called tuning curves because they show how patterns of neuronal firing are selectively "tuned" to different frequencies that vary from 0 to 50 kilohertz (kHz = 1000 hertz, or 1000 cycles per second; plotted on the x-axis). The minimum sound intensity that is required for a neuron to respond is shown in decibels (dB) on the y-axis. Thus, the best frequency for the neuron whose responses are plotted in red in Figure 1.27 is about 1.3 kHz. To get that cell to respond to another frequency, the sound will need to be louder. As another example, the purple line shows a different cell whose best frequency is about 6 kHz. FIGURE 1.27 Tuning curves of auditory neurons Graph of the responses of six different neurons (shown in different colors) to sounds of different frequencies and intensities. Arrows along the x-axis indicate the frequency that generated the best response from that neuron. (After N. Y. S. Kiang. 1965. Discharge Patterns of Single Fibers in the Cat's Auditory Nerve. MIT Press: Cambridge, MA.) View larger image Neuroimaging Modern perception researchers can now use other tools to understand how thousands or millions of neurons work together within the human brain. In many cases, we look at the results of these methods by making pictures of the brain that reveal its structure and functions. These methods can be collectively referred to as neuroimaging methods. Of course, neuroanatomists have long described the structure of the human brain, and neuropsychological studies of individuals with brain damage have taught us much about the functions of different parts of the brain. However, a great advance in recent decades has been the invention of neuroimaging methods that allow us to look at the structure and function of the human brain in healthy, very much living human observers. For example, electroencephalography (EEG) measures electrical activity through dozens of electrodes placed on the scalp (FIGURE 1.28A). EEG does not allow researchers to learn what individual neurons are doing or to pinpoint the exact area of neural activity. However, EEG can be used to roughly localize whole populations of neurons (FIGURE 1.28D) and to measure their activities with excellent temporal accuracy. Like a single behavioral measurement, a single EEG signal recorded for a single event is usually not terribly informative. If you want to know the brain's response to a prick of the skin or a flash of light against a background of millions of neurons firing for other reasons or for no reason at all, you will want to repeat the measurement many, many times. Three hypothetical response to the onset of a light are shown in FIGURE 1.28B. You then average all the responses aligned to the moment that the stimulus was present. The resulting averaged waveform is known as an event-related potential (ERP). Figure 1.28C shows the typical ERP from an experiment in which observers saw brief flashes of light. The EEGs for the 600 milliseconds (ms = 1/1000 sec) after each flash were averaged together to produce the waveform. On average, nothing much happens for the first 50 ms or so. Then there is a small positive deflection of the signal, followed by a larger negative deflection. (ERP researchers like to plot positive down and negative up. Don't ask.) These signals can tell us quite a bit about the temporal processing of a stimulus. FIGURE 1.28D shows the signal as a function of time and space. Where on the scalp are electrodes picking up a signal. In this case, the response to a light first appears at the back of the head, the home of "primary visual cortex" before flowing forward in the brain to later stages of visual processing (see Section 3.4). FIGURE 1.28 Electroencephalography (EEG) (A) Electrical activity from the brain can be recorded from the scalp by using an array of electrodes. (B) The activity from one electrode is quite variable, even if the same stimulus (perhaps a flash of light) is presented multiple times. (C) However, if many such signals are averaged, a regular pattern of electrically positive (P) and negative (N) waves can be seen. Electrical activity (voltage) is measured in microvolts (μV). (D) Different signals from different electrodes can be used to create a rough map of scalp topography of the activity elicited by a stimulus. These views, looking down at the brain from above, illustrate the typical changes one might see from 50 to 150 ms after the onset of a visual stimulus. Notice that the signal is initially focused over early visual cortex (50--100 ms), then shifts to more anterior areas of visual cortex (101--125 ms) and then begins to activate frontal cortex (126--150 ms). (A--C after S. M. Breedlove et al. 2010. Biological Psychology: An Introduction to Behavioral, Cognitive, and Clinical Neuroscience, 6th ed., Oxford University Press/Sinauer Associates: Sunderland, MA.) D Courtesy of Steven Luck View larger image A related method known as magnetoencephalography (MEG) (FIGURE 1.29) also provides good measures of neuronal timing while providing a better idea of where in the brain neurons are most active. MEG takes advantage of the fact that very small changes in local magnetic fields accompany the small electrical changes that take place when a neuron fires. MEG researchers use extremely sensitive devices to measure these tiny magnetic field changes. Why use EEG if you can have MEG's better spatial resolution? EEG recording is relatively simple and relatively cheap. MEG devices like that shown in Figure 1.29 use superconducting quantum interference devices (SQUIDs). These SQUIDs are much more expensive than the squid in Figure 24C. FIGURE 1.29 Magnetoencephalography (MEG) (A) What looks like a huge helmet contains superconducting magnets. (B) The output of MEG recording can be used to visualize activity in the brain. In this case, the individual saw pictures of objects. In this lateral view, "hotter" colors (yellows and reds) indicate more activity, and the "hot spot" at the back of the brain is the primary visual cortex. The superimposed floating squares represent the array of MEG detectors. A © Jim Thompson/Albuquerque Journal/ZumaPress; B Courtesy of Daniel Baldauf View larger image As noted above, one of the most exciting and important changes in neuroscience in the last generation has been the advent of methods that enable us to see the brain while it is still in its living owner's head. FIGURE 1.30A shows an actor viewing a standard X-ray of a skull; Figure 1.30B shows an example of the much richer images that magnetic resonance imaging (MRI) can produce. You can easily imagine the improvements in medical care that became possible once doctors could see a tumor or a blood clot without having to open the skull (see FIGURE 1.30B). FIGURE 1.30 Magnetic resonance imaging (MRI) Standard X-rays of the head do not reveal much about the brain. (A) Here actor, Amaka Umeh, portraying Hamlet at Stratford, Canada, contemplates an X-ray of the skull of Yorick (You remember: "Alas, poor Yorick, I knew him..."). (B) A magnetic resonance image, or MRI, shows considerably more detail than an X-ray, including a cancer in the cerebellum (dense black circle at lower right). A Amaka Umeh (Hamlet -- Stratford Festival, 2020). Creative direction by Punch & Judy Inc. Photography by David Cooper.; B Courtesy of Geoff Young, MD View larger image To produce MRI images of the brain, that brain and its owner are placed in a magnetic field powerful enough to influence the way the atoms spin. The physics is complicated and beyond the scope of our text, but by pulsing the magnetic field, it is possible to measure a signal that indicates the presence of specific elements in the tissue. If you ask about hydrogen in the brain, you are (mostly) asking about water, and your computer can use the hydrogen signal to reconstruct the structure of the water-rich tissue inside your head. For the study of the senses and, indeed, for the study of many topics in psychology, the most remarkable use of MRI technology is functional magnetic resonance imaging, or fMRI. With fMRI, we can see the activity of the living brain. Here the critical factor is that active brain tissue is hungry brain tissue. It needs oxygen. Oxygen and other supplies are delivered by the blood, so an active brain demands more blood. The result is that there is a blood oxygen level--dependent (BOLD) signal that can be measured by the MRI device. Instead of indicating the presence of water, the magnetic pulses and recording are used to pick up evidence of the demand for more oxygenated blood. There are some drawbacks. It takes a few seconds for the BOLD signal to rise after a bit of brain becomes more active, so the temporal resolution of the method is slow compared with EEG/ERP and MEG. The machines are noisy, making auditory experiments difficult. Moreover, acquiring and running these machines is expensive. Still, none of these drawbacks have prevented the method from revolutionizing the study of the brain. FIGURE 1.31 shows some of the results from a very basic fMRI experiment. The observer was watching a screen. A visual stimulus was turned on for 30 seconds and off for 30 seconds in alternation. Regions of the brain that are made more active by the presentation of a visual stimulus demand more oxygen supplies in response to the presence of the stimulus, so there is a difference between the BOLD responses to stimulus-on and stimulus-off conditions (with that lag of several seconds). Subtracting the stimulus-off fMRI signal from the stimulus-on signal reveals the difference in BOLD responses. Typically, these are shown as colors superimposed on a structural MRI of the brain. FIGURE 1.31 Functional MRI (fMRI) The warm colors show places where the BOLD signal was elevated by the presence of a visual stimulus. Blues show decreases in BOLD activity. The dense region of signal elevation at the back of the brain corresponds to the primary visual areas (the visual cortex) (see Section 3.4). Courtesy of Steve Smith, University of Oxford FMRIB Centre View larger image Positron emission tomography (PET) is an imaging technique in which a small amount of a safe, biologically active, radioactive material (a tracer) is introduced into the participant's bloodstream, and a specialized camera detects gamma rays emitted from brain regions where the tracer is being used most (Phelps, 2000) (FIGURE 1.32). The premise is similar to that of fMRI: to detect activity in neurons by looking for increased metabolic activity. The most common tracer used in perception experiments is oxygen-15 (15O), an unstable form of oxygen that has a half-life of only a little more than 2 minutes. Neurons that are most active in the brain should have the greatest requirement for oxygen, and thus the greatest amount of the tracer. Although PET is a somewhat inconvenient method because you have to inject a tracer, it has the advantage of being silent, which is helpful in studies of brain activity related to hearing. FIGURE 1.32 Positron emission tomography (PET) PET is a form of neuroimaging that uses positron-emitting radioisotopes to create images of biological processes in the living brain. In these images looking down from above, we see different patterns of glucose usage in the brain while an individual is performing different tasks. From M. E. Phelps. 2000. Proc Natl Acad Sci USA 97: 9226--9233. © 2000. National Academy of Sciences, U.S.A. View larger image Modern labs often use multiple methods in the same experiment or series of experiments. You might use behavioral methods to determine the nature of a particular perceptual capability, but such methods might take hours and hundreds or thousands of trials. If your observers can't spend that much time in the scanner, you might have them perform a shorter, stripped-down version of the study while being imaged. You might do yet another version of the task, specialized for the demands of EEG recording. By means of these "converging operations," you could build up quite a detailed picture of what a person perceives and how her brain gives rise to those perceptions. 1.4 Modeling as a Method: Math and Computation Learning Objectives By the end of this section, you should be able to: 1.4.1Explain how mathematical and computational models can be used to study perception. 1.4.2Describe the computational models covered in the chapter: efficient coding models, Bayesian models, artificial neural networks, and deep neural nets. Mathematics and computer programs have been used for a long time to better understand sensation and perception. Based upon their research with giant squid axons back in the 1950s, Hodgkin and Huxley developed a mathematical model to describe how action potentials in neurons are initiated and propagated. Weber, Fechner, and Stevens's Laws can also be considered to be mathematical models. Mathematical models use mathematical language, concepts, and equations to closely mimic psychological and neural processes with mathematical precision. In principle, mathematical models could be (and often were) worked out with pencil and paper. Modern computational technology has allowed us to expand such models into a new class of computational models that can take advantage of ever faster and more powerful computers to describe and understand sensation and perception in ways that would be impossible with manual techniques. Today's sophisticated models do more than simply describe how systems work; they help scientists better understand how neural circuits develop and organize to make sensation and perception work in the ways that they do. In almost every chapter of this book, you will learn that understanding how a sense works depends on understanding how that sense develops as the organism gains experience in the world. This is because experience will dramatically change the actual structure of the nervous system. If you recall our earlier claim that "the mind is what the brain does," you can imagine that every time you learn something, you are changing your brain. Computational models are approaching a state where they may be able to mimic and explain those changes. Here we provide very brief introductions to four types of computational models. Computational Models: Probability, Statistics, and Networks To begin, imagine for a moment what the visual world would look like if it was just noise; something like FIGURE 1.33A. Or, imagine that all your ears heard was the hiss of a white noise machine that some people use to sleep better. Of course, the real world does not deliver only random noise to your senses (FIGURE 1.33B). The real world is structured and predictable---not completely predictable, but, for instance, in Figure 1.33B, if you know that pixel X is green, you could guess that the pixel next to it is green, too. You might not always be right, but you won't be guessing either. An effective perceptual system learns about predictability and structure through experience. FIGURE 1.33 Predictable pixels (A) In a field of random noise, knowing if one spot is black tells you nothing about its neighbor. (B) The real world is more structured, redundant, and predictable. If you know that one pixel is blue, it is a good (but not perfect) bet that its neighbor is blue. A © Pawel Michalowski/Shutterstock.com; B Courtesy of Jeremy Wolfe View larger image Efficient coding models are designed to discover predictability and structure. Efficient coding models use precise ways to mathematically define "information" (Shannon, 1948). The math quantifies the idea that information tells you something that you do not already know. The more predictable something is, the less information it conveys. An efficient system should not spend a lot of resources on inputs that are predictable and/or redundant (Attneave, 1954; Barlow, 1961, 2001). So, if pixels 1, 2, 4, 5 are green, an efficient system would not spend a lot of effort encoding the greenness of pixel 3. These principles are hardwired into your phone, so your carrier can allocate as little data as possible to your conversations. In computational models of sensation and perception, efficient coding models discover predictability (structure) in sensory input and organize to economically encode the world. For example, applying principles of efficient coding helps to solve many of the difficult questions about how speech is perceived (Kluender, Stilp, and Llanos, 2019). Bayesian models are like efficient coding models, with the additional property that these models actually attempt to build a model of the world, or at least an estimate of the process that generates the energy that arrives at your senses. These models use Bayesian statistics, which is different from the Fisherian statistics that you may have learned. Fisherian statistics treat every new piece of data as independent of previous data---the common example being that a given flip of a coin does not bias the result of the next coin flip; this is the basis of the t-tests and ANOVAs of your statistics courses. Bayesian statistics, however, assume that earlier observations should bias expectations for future events. If pixels 1, 2, 3, 4, and 5 are green, what color do you think pixel 6 will be? Because the world is generally predictable, Bayesian statistics can be useful in understanding how perceptual systems adapt with experience (Geisler, 2011). Bayesian models provide what is called predictive coding, and they assume that the brain builds a model about the environmental causes of the sensory inputs it receives. The model infers these causes by making a "best guess," or prediction, about inputs at each point in time and then evaluating whether the predicted input corresponds with the input that was actually received. If not, the system will attempt to reduce this mismatch, or prediction error, by adjusting its prediction about the state of the environment and adapting its model accordingly (Van de Cruys, Van der Hallen, and Wagemans, 2017). Bayesian models have been especially useful in helping to understand visual perception, as will be discussed in Section 4.4. Artificial neural networks, inspired by biological neural networks, provide another computational framework to better understand how sensory systems develop based on experience. Sometimes called connectionist models, artificial neural networks are comprised of layers of heavily interconnected computational units. These units are analogous to neurons massively connected to one another through their axons, dendrites, and synapses. The strength of the connections between units can increase or decrease, depending upon how they contribute to the network's success (Rumelhart, McClelland, and PDP Research Group, 1986a, 1986b). This is akin to the strength of a synapse changing with experience. Some neural network models "learn" based on feedback---that is, being told whether or not they are correct---while others function without feedback. If a model is given feedback when it is right or wrong, it is a "supervised" model; models without this feedback are "unsupervised." Neural network models typically require multiple cycles of taking inputs and producing outputs before they settle into a best solution. It can be interesting to compare this training of a computer with the time course shown by humans as they learn the same information. Deep Learning The newest forms of neural network models are deep neural networks (DNNs), also called deep neural learning models, or, simply, deep learning. Deep learning models can have many layers of units (or nodes) with millions of connections, harnessing the speed and massive memory capacities of modern computers. Such networked models are particularly good at taking vast amounts of information and classifying it into categories. One huge success has been in the field of object recognition. As Section 4.5 will discuss, DNNs now routinely take as input millions of images and learn to identify all the dogs, cats, tomatoes, and so on, accurately labeling thousands of different types of objects (Krizhevsky, Sutskever, and Hinton, 2012; Kriegeskorte, 2015). This is the technology behind the artificial intelligence boom that has brought you (for better or worse) devices like Amazon's Alexa and the facial recognition software used by many police and security agencies. If we ever have fully autonomous cars, DNNs will be an important part of their "brains." Deep learning technology is on the verge of transforming tasks like those performed by expert radiologists because a network that can be trained to find cat videos can be trained to find cancer, too (Mendelson, 2018; Borstelmann, 2020). The ability of DNNs to perform like humans leads us to wonder if they are in fact models of how humans perform the task. They might be, or the DNN might be doing the task quite differently. A jet plane, for example, is not a great model of how a sparrow or a bumblebee flies. The difficulty is that, like the brain, these models are very complex and, like the brain, it is not a trivial task to figure out how they do what they do (Kriegeskorte and Douglas, 2018). This can be a problem when DNNs are used in the real world. For example, if a DNN was really going to be the "expert", looking for cancer in an x-ray, we would want that "expert" to be able to explain how and why it made a particular decision (Handelman et al., 2018). At present, that is not something that DNNs are good at. Still, these computational tools are making rapid progress and it safe to predict that future sensation and perception texts will have more to say on this topic.