SP - Chapter 3.pdf

Chapter 3 Iruka Maria Toro, Your Soul Has Become an Invisible Bee, 2014  Spatial Vision: From S po ts to Stripes Questions to Contemplate - Think about the following questions as you read this chapter. By the chapter's end, you should be able to answer and discuss t...

Chapter 3 Iruka Maria Toro, Your Soul Has Become an Invisible Bee, 2014  Spatial Vision: From S po ts to Stripes Questions to Contemplate - Think about the following questions as you read this chapter. By the chapter's end, you should be able to answer and discuss them. How do the images formed on the retina reach the brain and enable us to see t h e world? How do we see things "right side up" when the image on our retina si upside down? What can a baby see? n Chapter 2 we learned that the macroscopic structures of the human eye function essentially as a biological camera: The iris regulates the amount of light entering the eyeball. The cornea, lens, and aqueous and vitreous humors focus the light rays so that a clear image is formed on the retina. The rod and cone photorecep- tors capture this image in a way that is roughly analogous to the way the film in a camera captures photographic images. It is here, however, that the analogy between visual system and camera ends. Cameras take pictures. Visual systems see, and seeing leads to actions. How do we get from an image of the world in front of us to an interpretation of that wordl— what is out there, where it is, and what we can do with it? The process starts in the eyeball itself, where the postreceptoral layers of the retina translate the raw light array captured by the photoreceptors into the patterns of spots surrounded by darkness, or vice versa, detected by the ganglion cells (see Figure 2.20). As we discussed in Chapter 2, this retinal translation helps us perceive the pattern of light and dark areas in the visual field, regardless of the overall light level (e.g., it enables us to see almost as well at dusk as we can at noon). In this chapter, we follow the path of image processing from the eyeball to the brain (FIGURE 3.1). Ganglion cells in the retina respond preferentially to spots of light (Figure 3.1C, top). As we will see, neurons in the cerebral cortex prefer lines, edges, bars, and stripes (Figure 3.1C, bottom). Furthermore, this portion of visual cortex is organized into thousands of tiny microprocessors, each responsible for determining the orientation, width, color, and other characteristics of the scene in one small portion of the visual field. In Chapter 4, we will continue this story by examining how other parts of the brain assemble the outputs from these minicom- puters to produce a coherent representation of the objects whose reflected light started the photoreceptors firing in the first place. 58 Chapetr 3 Spatial Vision: From Spots to Stripes (A) (C) Left Parvocellular (ven Right Magnocellular (dorsal) tral) system system Opcit Inferior temporal nerve Ve n t r a l Parietal Optic - intraparietal (7a) Medial chiasm superior temporal (5a) Optic - tract Lateral - V5 (MT) geniculate Area nucleus 19 Optic - V4 radiation V3 Str iat e cortex Area 18 B )( Visual association. V2 cortex Area 17 V1 Layer V I Layer IV Lateral geniculate Magnocellular Parvocellular Higher-order Primary nucleus visual association visual cortex cortex (striate cortex) Retina M(large) ganglion P (small) ganglion cells FIGURE 3.1 Cortical visual path- ways (A) The basic organization of the primary visual pathway from eyeball to Color Motion Stereopsis Form Specific form 3.1 Visual Acuity: Oh Say, Can You See? The King said, "I haven't sent the two Messengers, either. They're both gone to the town. Just look along the road, and tell me fi you can s e e either of them." "I see nobody on the road," said Alice. "I only wish I had such eyes," the King remarked ni a fretful tone. "To be able to see Nobody! And at that distance, too!" —Lewsi Carroll, Through the Looking Glass Since we'll be talking ni this chapter about how the visual system codes images ni veres oforiented stripes, lets startybdetermining ust w oh wleevse snpies illumination between the stripes and the background-is very low. In addition ot  31. Visual Acuity: Oh Say, Can You See? 59 FIGURE 3.2 A visual acuity self-test Focusing on the central Xat different distances reveals the farthest point at which you can distinguish the ori- entation (horizontal versus vertical) of the lines. The steps for this simple t e s t of your visual resolution acuity are detailed in the text. setting the boundary conditions for how well we should expect the visual system acuity The smallest spatial detail that to be able ot perform, we wil use this section ot introduce some important jargon can be resolved at 100% contrast. that we'll need in the rest of the chapter. visual angle The angle subtended by an object at the retina. Get atape measure, prop your textbook up, and, while looking at the Xni the middle of FIGURE 3.2, back up until you cannot tell the orientation of the black and white stripes. Measure how far your eye is from the page. Now walk forward a oyd bar nda enobgriht consintg fo ebYiouinjust ycompleted ouet ure uoyafastnac(butes notshei terribly yournitgaccurate) het a stringsoafdyourwohch nieuldesmeasurement i wn visual resolution acuity, or simply acuity. Eye doctors specify acuity in terms like 20/20 (more about this ni amoment), but si hers. Vsei slangiedybhte met wen htey ngichapetr 2ebresovled rf a da onto farleft nsig ito,ed peanrignohte redyt nif gestom angle. Acycle si simply one repetition of a black stripe and awhite stripe (each of the two gratings in Figure 3.2 has 25 total cycles). T o calculate the visual angle of your resolution acuity (Figure 3.3), divide the size of the cycle in Figure 3.2 (which is 2 millimeters (mm)) by the viewing distance at which you could just barely make out the orientation of the gratings (average your first and second measurements to get a rough estimate of this distance), and then take the arctangent of this ratio. Under ideal conditions, humans with very good vision can resolve gratings like those in Figure 32. when one cycle subtends an angle of approximately 1 minute of arc 1( arc minute, or 0.017 degree). As a rough rule of thumb (see Figure 2.13), 1centimeter (cm) ~ 1degree (60 arc minutes) at aviewing distance of 57 cm. fI the size of the just resolvable cycle were 1cm, you would need ot back up ot about 3.42 meters (57 cm ×60 = 3420 cm) ot eb at hte acuity limit. 1 cycle Visual angle / Viewing distance (at resolution limit) FIGURE 3.3 Visual angle Shown here si the angle size of one cycle of a grating at the retina.  60 Chapter 3 Spatial Vision: From Spots to Stripes sine wave grating A grating with a This resolution acuity represents one of hte fundamental limits of spatial vision: sinu soi dal luminance pr of ile, as s hown in Fgiure 4A3.. ti si the finest high-contrast detail that can be resolved. The limit si determined primarily yb hte spacing of photoreceptors ni the retina. To see why, imagine that wer'e projecting hte sine wave gratings shown ni FIGURE 34.A onto hte retina. The light intensity ni such gratings varies smoothly and continuously across each IURE 34.B, ni which intensity changes cycle (unlike hte gratings ni Figure 32. and FG abruptly from black ot white and then back ot black). However, the visual system "samples" the grating discretely, through the array of receptors at the back of hte retina (in this respect, the eye si more like adigital camera than like a traditional camera that uses film). fI the receptors aer spaced such that the whitest and blackest parts of the grating fall on separate cones (FIGURE 3.4C), we should be able ot make out the grating. But if the entire cycle falls on a single cone (FIGURE 3.4D), w e wil se nothing but a gray field. (A) (B) Stimulus -推 (C) Cones (D) FIGURE 3.4 Grating patterns (A) A sine wave grating. (B) A square wave grating. (C) The stripes of the sine wave grating are wider than the pho- toreceptors (pink circles in the top panel), and the Perception grating can be reconstructed vertically. (D) The stripes of the sine wave grating are narrower than the photoreceptors, so both black and white bars will fall inside a single receptor (top panel), resulting ni a uniform gray field (bottom panel).  31. Visual Acuity: Oh Say, Can You See? 61 Cones in the fovea have a center-to-center separation of about 0.5 minute of arc (0.008 degree), which fits nicely with the observed acuity limit of 1 minute of arc (remember that we need two cones per cycle, one for the dark bar and one for the bright bar, to be able to perceive the grating accurately). Rods and cones in the periphery are packed together less tightly (recall that in the periphery, rods are physically more tightly packed (denser] than cones, as shown ni Figure 2.12), and here many receptors converge on each ganglion cell. As a result, visual acuity is much poorer in the periphery than in the fovea. Visual acuity ni peripheral vision is not uniform—it falls of more rapidly along the vertical midline of the visual field than along the horizontal midline. This si known as horizontal and vertical asymmetry. Thus, if you fix your eyes on one point, you have (slightly but measurably) better acuity 5degrees left or right than you do 5degrees up or down. W e also have better acuity a fixed distance below hte midline of the visual field than above. This si known as vertical meridian asymmetry (Abrams, Nizam, and Carrasco, 2012). Interestingly, although visual acuity falls off rapidly ni the visual periphery, it is not the major obstacle to reading or object recognition. The real problem in the periphery is known as visual crowding-the deleterious effect of clutter on peripheral object recognition (Levi, 2008; Whitney and Levi, 2011). Objects that can be easily identified in isolation seem indistinct and jumbled when surrounded by other objects (FIGURE 3.5). Luckily, we are able to make eye movements in order to foveate and scrutinize individual objects in clutter. There is another surprising difference between central vision and peripheral vision: central vision is considerably slower than peripheral vision. Recent work (Sinha et a,l. 2017) suggests that peripheral cones respond about twice sa quickly to light as do foveal cones (30 versus 60 ms). Foveal cones have longer axons than visual crowding The deleterious peripheral cones in order to allow dense packing in the central fovea, and the effect of clutter on peripheral object longer axons transmit slow signals better than fast ones (Masland, 2017). The slow recognition. (A) (B) - P + + + ハ| WP C 各0 → → cross) (left cross), line orientation (middle cross), or letter (right FIGURE 3.5 Visual crowding (A) Visual crowding occurs ni above it, but it si difficult or impossible ot identify the same natural scenes. When fixating on the bull's-eye near the shape, line orientation, or letter when it is in the middle of a construction zone, note that it si difficult or impossible to recog- group below the cross. However, it is easy to do when side of the road, sim- nize that there is a child on the left-hand looking at the patterns directly. Crowding in peripheral vision However, ti is ply because of the presence of the nearby signs.right -han d side. impairs the ability to recognize objects, but it does not make them on the relatively easy to recognize the child the shape disappear. identify (B) While fixating on each cross, it is easy to 62 Chapter 3 Spatial Vision: From Spots to Stripes -ニ Stroke ya aolw foveal conesot increase their reliability yb integrating their response m inputs over a longer time. A Visit to the Eye Doctor ite ey doctors dont' describe acuity ni terms fovisual angles and cycles. The last m E you visited your eye doctor, she may have asked you to read letters, decreasing hte size of the letters until you made several errors. Then she may have told you that your visual acuity was 20/20 if your vision was good, or 20/40 fi you needed glass- es, ro possibly 20/10 fi you could read the smallest letters no the eye chart. Thsi FIGURE 3. 6 Snellen Visual Acuity method for designating visual acuity was invented ni 1862 by aDutch eye doctor, Char t This chart is a familiar test for Herman Snellen (1834-1908). Snellen constructed a set of block letters for which visual acuity used since the late 1800s. the letter as a whole was five times as large as the strokes that formed the letter The letter size is five times the stroke size. (FIGURE 3.6). Note that the resulting patterns are reminiscent of the gratings ni Letters get progressively smaller down the lines of a Snellen chart, but this ratio Figure 3.2. He then defined visual acuity as follows: remains the same. (distance at which a person can just identify the letters) (distance at whcih aperson whti normal vision can just identify hte leters) In later adaptations of the Snellen test, the viewer was positioned at a constant distance of 20 feet, and the size of the letters, rather than the position of the viewer, was altered. So normal vision came to be defined as 20/20. To relate this measure back to visual angle, a 20/20 letter si designed to subtend an angle of 5arc minutes (0.083 degree) at the eye, and each stroke of a 20/20 letter subtends an angle of 1arc minute (the familiar 0.017 degree). Thus, fi you can read a 20/20 letter, you can discern detail that subtends 1 minute of arc. If you have to be at 20 feet to read a letter that someone with normal vision can read at 40 feet, you have 20/40 vision (worse than normal). Although 20/20 si often considered the gold standard, most healthy young adults have an acuity level closer to 20/15. Note that while the acuity for stripes and letters si quite similar for individuals, the w t o types of stimuli provide different cues and are subject to different constraints. Indeed, ni some patients with amblyopia, acuity with Snellen letters may be much more affected than acuity with gratings. More Types of Visual Acuity How should we define the keenness of sight? The term visual acuity specifies a spatial limit. So far, we've discussed two forms of visual acuity: (1) the finest stripes that can be resolved (sometimes referred to as the minimum resolvable acuity) and amblyopia A developmental disorder (2) the smallest letter that can be recognized (the minimum recognizable acuity). characterized by reduced spatial vision ni an otherwise healthy eye, even with Over the centuries, there have emerged various ideas about how ot define, measure, ob okm A r seayl ye acvite eror. and specify visual acuity. TABLE 3.1 lists four of the most common definitions and we briefly discuss them next. TABLE 3.1 Summary of the different forms of acuity and their limits Type of acuity Measured Acuity (degrees) Minimum visible Detection of a feature 0.00014 Minimum resolvable Resolution of two features 0.017 Minimum recognizable Identification of a feature 0.017 Minimum discriminable Discrimination of a change ni a feature 0.00024  3.1 Visual Acuity: Oh Say, Can You See? 63 MINIMUM VISIBLE ACUITY Minimum visible acuity refers to the smallest ob- ject that one can detect. Under ideal conditions, humans can detect a long, dark wire (like a cable of the Golden Gate Bridge) against a very bright background (like the sky on a bright, sunny day) when they subtend an angle of just 0.5 arc second (about 0.00014 degree). Minimum visible acuity is so small for two rea- sons. First, the optics of the eye (described in Chapter 2) spread the image of the thin line, making it much wider on the retina; and second, the fuzzy retinal im- age of the line casts a shadow that reduces the light on a row of cones to a level that is just detectably less than the light on the row of cones on either side. A-l though we specify the minimum visible acuity in terms of the angular size of the target relativeotsi backgroedyba si. m yi ot sdivine acche intensity fi het ability to discern small changes in contrast, rather than a spatial limit per se, and is not used clinically. MINIMUM RESOLVABLE ACUITY Minimum resolvable acuity is what you just t een negihbe nig obiet measuer in reseot Anciealest angual separation bew observer to resolve double stars. However, today the minimum resolvable acuity is much more likely to be assessed by determining the finest black and white stripes that can be resolved. Under ideal conditions (e.g., high contrast and luminance), humans with very good vision can resolve black and white stripes when one cycle subtends an angle of approximately 1 minute of arc (0.017 degree). This minimum resolvable acuity represents one of the fundamental limits of spatial vision: it is the finest high-contrast detail that can be resolved. In foveal vision, the limit is deter- mined primarily by the spacing of photoreceptors in the retina. MINIMUM RECOGNIZABLE ACUITY Minimum recognizable acuity refers to htbey Herman anch raSnellen szie fehand te eat feat ury ha t i nt rodu cec o g his colleagues, as discussed earlier. n i ze r ident u r y h s i MINIMUM DISCRIMINABLE ACUITY Minimum discriminable acuity refers to the angular size of the smallest change in a feature (e.g., a change in size, position, moreinat dsicrm i niane audi somae eris hte ft sended exercive positions of two features. Our visual system is very good at telling where things are relative to each other. Consider two abutting horizontal lines, one slightly higher than the other. The smallest misalignment that we can reliably discern is known as Vernier acuity, named after the Frenchman Pierre Vernier (1580-1637). Vernier's scale was based on the fact that humans are very adept at judging whether nearby nile eve lied eydianst. ein den ovil. nidy uedsni cordisions chnies acuity may be just 3 arc seconds (about 0.0008 degree)! This performance is even more remarkable when you consider that it is about ten times smaller than even the smallest foveal cones. Consider that the optics of the eye spread the image of a thin line over a number of retinal cones and that the eyes are in constant motion, and this performance appears even more remarkable. Vernier acuity is not the most remarkable form of hyperacuity. Guinness World Records (2005) describes the "highest hyperacuity" as follows: "In April 1984, Dr. Dennis M. Levi [yes, that's one of the authors of this book] repeatedly identified the relative position of a thin, bright green line within 0.8 seconds of arc (0.00024 64 Chapter 3 Spatial Vision: From Spots to Stripes Ts hi si equ v i ae l nt t o ads ipa lcm e n e t o f o sme. 2 05 inches 6( mm)ta a e. degre of 1mile (1.6 km)." distance st Stripes Acuity for Low-Contra ontrast details htat ew can re- discussing hte tiniest high-cngs pot now, wve'e tbheaent high U solve. We learned -contrast sine wave grati can be distinguished fromyba uniform gray field, sa olng as adj are separated acent pairs oflight or dark stripesrast at least 1arc minute fo visual angle. Butaer w hat happens if hte cont fohte stripees si reduced-that is, fi het light stripes made darker and the dark stripes light r? , when eh was working ofr hte This was hte question asked yb Oto Schadseineniwa1v9e56grati RCA Corporation. Schade showed people of the gratings with different spatial frequencies and had them adjust the cont rast ngs until they could just be detected. Spatial frequency refers ot the number spoafce (degreaepatte times rn, such sa a sine wave grating, repeats a( cycle) ni agiven unit of f visual angleof. vi sual angle). Thus, ti si measured as number of cycles per degree ovisual angle betFow r example, c away, hte een each ur book from about 120 m fi you view yostripes a relatively o l w spatial pair of white ni FIGURE 3.7A shows a grating with 0.25 degree, os the frequency-about 2 cycles per degree. FIGURE 3.7B is about 3.7C spatial frequency of this grating is 1/0.25 = 4 cycles per degree, and FIGURE spatial frequency The number of illustrates a relatively higher spatial frequency (about 8 cycles per degree). Intuitively, you might think that the wider the stripes (that is, the lower the spatial frequency), the easier it would be ot distinguish the light stripes from the dark stripes. But this si not what Schade found. H e, and later Fergus Campbe ll nd Dan a Green (1965), demonstrated that the human contrast sensitivity function (CSF) si cycles per degree The number of shaped like an upside-down U, as shown ni FIGURE 3.8. We obtain the units for the grating cycels per degree fovsiual left side of the y-axis —the observer's contrast sensitivity— by taking the reciprocal the contrast threshold, graphed on the y-axis on the right. For example, for : contrast secting unf hte sensivi, o1fcycle/degree grating (the x-axis) ot be just distinguishable from uniform gray, the te confrequency hspatial ets thesis depecisnoof trites ,set have apatra ofagilht stripe reflects 1010 phootns and apacth oaf (size) of the stimulus. dark stripe should reflect 990 photons). The contrast, C, of a grating si generally specified according to the definition described by the first American to win the Nobel Prize, the physicist Albert Michelson. (He won the Nobel Prize ni 1907 for (A) Low (B) M e d i u m (C) High FIGURE 3.7 Different spatial frequencies and high spatial frequencies. Sine wave gratings illustrating low, medium,  3.1 Visual Acuity: Oh Say, Can You See? 65 1000 0.1 FIGURE 3.8 The contrast sensitivity function and the window of visibility The y-axis on the left is the observer's con- Invisible trast sensitivity; its units are obtained by taking the reciprocal of the object's contrast at threshold (the y-axis on the right), determined using the Michelson equation described ni the text. Any objects whose spatial frequencies (x-axis) and contrast thresholds (right Contrast sensitivity 100 y-axis) fall within the yellow region will be visible. Those outside Contrast (%) the yellow region are outside the window of visibility. The red line Visible delimits the contrast sensitivity function-the threshold between seeing and not seeing. 10 10 100 0.1 1 10 t 100 60 Spatial frequency (cycles/ degree) his work on the measurement of the speed of light.) According to the Michelson definition, C=(Imax - Lmin)/(Lmax +Lmin), where Imax and Lmni are the maximum and minimum luminance, respectively. In our example, C = (1010 - 990)/(1010 + 990) = 0.01 or 1% (0.01 x 100). The reciprocal of this threshold is 1/0.01 = 100, so this is the point plotted on the red CSF line in Figure 3.8 for this spatial frequency. Note that a contrast of 100% corresponds to a contrast sensitivity value of 1. The CSF reaches this value on the far-right side of the curve in Figure 3.8, at about 60 cycles/degree. Sixty cycles/degree corresponds to a cycle width of 1 minute of arc, the resolution limit we measured previously for high-contrast stripes, which, recall, is determined primarily by cone spacing. The falloff in the CSF on the other side of the curve cannot be explained by cone spacing or by limitations in the optics of the eye. Instead, this part of the function must be a result of neural factors, which we will discuss later in the chapter. You can visualize your own CSF using FIGURE 3.9. Here we see a sinusoidal grating whose contrast increases contin- uously from the top of the figure to the bottom and whose spatial frequency increases continuously from the left side of the graph to the right. If you view the figure from a distance of about 2 meters, you will notice the inverted U shape where the grating fades from visibility ot invisibility. If you bring the book closer to your eye, you should be able to see the stripes on the right side of the figure going farther up, whereas the tops of the stripes on the left side will become less distinct. There are many factors that influence the exact form of the CSF. These include the adaptation level of the eye (FIGURE 3.10A), the temporal modulation of the targets (i.e., how it varies over time; FIGURE 3.10B), and the age (FIGURE 3.10C) and refractive state (e.g., nearsightedness) of the individual. Why Sine Wave Gratings? One answer to this question is that, although "pure" sine wave gratings may be rare in the real world, patterns of stripes with FIGURE 3.9 Visualizing your contrast sensitivity function (CSF) When viewed from a distance of about 2 meters, a grat- more or less fuzzy boundaries are quite common: think of trees ing modulated by contrast (vertically) and by spatial frequency in a forest, books on a bookshelf, and a map of Manhattan (horizontally) appears in the inverted U shape of the CSF.  66 Chapetr 3 Spatial Vision: From Spots to Stripes (B) Temporal modulation (A) Adaptation level 500 Photopic 1 200 Contrast sensitivity 100 Contrast sensitivi 50 Scotopic 16 10 20 10 4 6 5 16 2 100 60 100 0.2 0.5 1 2 5 10 10 20 50 Spatial frequency (cycles/ degree) Spatial frequency (cycles/ degree) (C) Age 1000 Contrast sensitivity 100 -20s -60s 10 * 20s. -80s 60s FIGURE 3.10 Factors influencing the contrast sensitivity function (CSF) 80s The shape and height of the CSF is influenced by a wide variety of factors, such as adaptation level (A); temporal modulation (B), where each curve represents the CSF at a different temporal frequency (1, 6, and 16 Hz); and age (C), where 0.1 1 10 100 each curve represents the CS at a different age (20s, 60s, and 80s). Spatial frequency (cycles/ degree) (the latter includes a pattern of horizontal stripes superimposed on a pattern of vertical stripes). Furthermore, the edge of any object produces a single stripe, often blurred by a shadow, in the retinal image. On a larger scale, the visual system appears to break down real-world images into a vast number of components, each of which is, essentially, a sine wave grating with a particular spatial frequency. This method of processing is analogous to the way ni which the auditory system deals with sound. French mathematician Joseph Fourier (1768- 1830) developed analyses that help modern perception scientists ot better describe how complex sounds such as music and speech, complex head motions, and complex images can be decomposed into a set of simpler components-that is, sine waves. Any complex sound can be broken down into individual sine wave components through this process, which is called Fourier analysis. Fourier analysis si apowerful mathematical tool that si used ni many research fields. As we wil se ni Section 12.4, Fourier analysis si used extensively by vestibular and spatial orientation researchers. Vision researchers use Fourier analysis ot describe images ni terms fo Fourier analysis A mathematical their spatial frequencies —that is, as the cycle of changes from light and dark across procedure by which any signal can be space (FIGURE 3.11, left column). Images can be broken down into components separated into component sine waves that capture how often changes from light ot dark occur over aparticular region ni at different frequencies. Combining the sine waves (Fourier synthesis) will repro- space, called spatial frequencies. Spatial frequencies are defined sa hte number of duce the original signal. these light/dark changes across 1degree of aperson's visual field. Thus, ni vision, hte phase The position of me a grating rel- units of spatial frequency are cycles per degree of visual angle. Similarly, acomplex ative to a fixed position asured in sound can eb broken down into aset of sine wave pure tones. Conversely, complex degre es, where one complete cycle is images can be constructed by adding together sine waves of different amplitudes 360 degrees. and phases (Fourier synthesis). The phase of the sine wave is its position relativeot  32. Retinal Ganglion Cells and Stripes 67 FIGURE 3.11 Fourier analysis (Left column) Sine waves of different spatial frequencies and amplitudes. From top to bottom: First row, the "fundamental" spatial frequency (f) with amplitude a; amplitude decreases as frequency increases (second row, 3f, a = 1/3; third row, 5f, a = 1/5; fourth row, 7f, a = 1/7; fifth row, 9f, พา พWW a = 1/9). (Right column) Illustration of how a square wave can be constructed by adding a series of sine waves with the appropriate ampli- tudes and phases. 1- f + 3f + 5f + 7f +1 1- 9f f + 3f + 5f + 7f+ 9f +1 -1| 0 2 4 6 8 10 : Time All possible harmon ics present -1 Time a fixed marker. Phase is measured in degrees, with 360 degrees of phase across one period, like the 360 degrees around a circle. For example, Figure 3.11, right column, illustrates how a simple square wave can be constructed by adding a series of sine waves with the appropriate amplitudes and phases. Well return to this idea later in the chapter. Of course, most visual images consist of features with different sizes and orientations; that is, they are two-dimensional. Just how this is done is beyond the scope of this chapter. For now, rest assured that scientists don't use sine wave gratings just because they're convenient to manipulate in experiments (although they do make very nice stimuli). 3.2 Retinal Ganglion Cells and Stripes In Section 2.4, we learned that retinal ganglion cells respond vigorously to spots of light. As it turns out, each ganglion cell also responds well to certain types of gratings, or stripes. FIGURE 3.12 shows how an ON-center retinal ganglion cell responds to gratings of different spatial frequencies. When the spatial frequency of the grating is too low, the ganglion cell responds weakly because part of the  68 Chapter 3 Spatial Vision: From Spots to Stripes A () Low frequency yields weak resp onse. fat, br,igdhatmbparingof ththee grating lands ni the inhibitory su.r round cell's response. Similarly, when the spatial frequency is oto high, hte ganglion cel responds weakly because both dark and bright stripes fal withinhet receptive-field center, washing out the response. But when te spatial frequency si just right, with abright a filing hte + hcenter and dark bars filling the surround, the cell responds vigorously. Thus, these retinal ganglion cells are "tuned" ot spatial frequency: each cell acts like a filter, respond- (B) Medium frequency yields strong response. ing best ot a specific spatial frequency that matches its receptive-field size and responding less ot both higher and lower spatial frequencies. Christina Enroth-Cugell and John Robson (1984) were the first to record the responses of retinal ganglion cells ot sine wave gratings. In addition to showing that these cells respond vigorously to gratings of just the right size, the investigators discovered that responses depend on (C) High frequency yields weak response. the phase of the grating— its position within the recep- tive field. FIGURE 3.13 illustrates how an ON-center retinal ganglion cell might respond to a grating of just the right spatial frequency (that is, a bar width about the size of the receptive-field center) in four different + →- phases. Note, however, that ganglion cells located ot the left or right of the target cell shown in Figure 3.13 might respond to the 90-degree and 270-degree phases FIGURE 3.12 ON-center ganglion cell response to (Figures 3.13B and 3.13D), but not to the 0-degree and The response (right) of an ON-center retinal ganglion 180-degree (Figures 3.13A and 3.13C) phases, which si cell to gratings of different spatial frequencies (left). (A) Too low: The why the visual system as a whole is able to see all four ganglion cell responds weakly because part of the stimulus (bright phases equally well. white) lands in the inhibitory surround. (B) "Just right": The cell responds vigorously when a bright bar fills the field's center and dark bars fill the surround. (C) Too high: Bright and dark stripes both fall FURTHER DISCUSSION of ON- and OFF-center within the center of the field, washing out the cell's response. ganglion cells can be found in Section 2.4. (A) 0° - Positive response (B) 90° - No response (C) 180° - Negative response (D) 270° - No response FIGURE 3.13 The response of a ganglion cell depends on bar, and similarly for the surround. There si thus no net difference The image depicts the response of an ON-center ret- between the light intensity ni the center and surround, and the inal ganglion cell to four different phases of an optimally sized cell's response rate wil not change from its resting rate. (C) Asec- grating. (A) When grating size and phase are optimal (i.e., alight ond 90-degree phase shift puts the dark bar ni the center and the bar fills the receptive-field center and dark bars fill the surround), () A light bars ni hte surround, producing a negative response. D this ON-center ganglion cell responds vigorously, increasing its third 90-degree shift returns us ot the situation after the first shift, firing rate. (B) fI the grating phase is shifted by 90 degrees, half with the overall intensities ni the center and surround equivalent the receptive-field center si filled by alight bar and half by adark and the cell therefore blind to the grating.  3.3 The Lateral Geniculate Nucleus 69 filter An acoustic, electrical, elec- 3.3 The Lateral Geniculate Nucleus tronic, or optical device, instrument, The axons of retinal ganglion cells synapse in the two lateral geniculate nuclei compu ter progra m, or neuron that allows the passage of some range of (LGNs), one in each cerebral hemisphere. These nuclei (clusters of similar neu- parameters (e.g., orientations, frequen- rons) act as relay stations on the way from the retina to the cortex (see Figure 3.1). cies) and blocks the passage of others. FIGURE 3.14 shows that the LGN of primates is a six-layered structure, a bit like a lateral geniculate nucleus (LGN) A stack of pancakes that has been bent in the middle (geniculate means "bent"). The structure in the thalamus, part of the neurons in the bottom two layers are physically larger than those ni the top four midbrain, that receives input from the retinal ganglion cells and has input and layers; for this reason, the bottom two are called magnocellular layers, and the top output connections to the visual cortex. four are called parvocellular layers (magno- and parvo- are Latin for "large" and "small," respectively). The two types of layers also differ in another, more important, magnocellular layer Either of the bot- tom two neuron-containing layers of the way: the magnocellular layers receive input from M ganglion cells n i the retina, lateral geniculate nucleus, the cells of and the parvocellular layers receive input from P ganglion cells (see Section 2.4). which are physically larger than those in The layers differ in more than the size of the cells. Studies in which magnocellular the top four layers. and parvocellular layers are chemically lesioned indicate that the magnocellular parvocellular layer Any of the top four neuron-containing layers of the pathway responds ot large, fast-moving objects, and the parvocellular pathway si lateral geniculate nucleus, the cells of responsible for processing details of stationary targets. This distinction si interesting which are physically smaller than those because it shows that the visual system splits input from the image into different in the bottom two layers. types of information. koniocellular cell A neuron located Even more splitting takes place between the layers. There, we find the layers between the magnocellular and parvo- cellular layers of the lateral geniculate consisting of koniocellular cells (konio is Greek for "dust"; these little cells were nucleus. This layer is known as the ignored for many years). The koniocellular layers are in the spaces between the koniocellular layer. magno and parvo layer (Figure 3.14) (Casagrande et al., 2007; Nassi and Callaway, 2009; Szmajda, Grünert, and Martin, 2008). Each koniocellular layer seems to be involved in a different aspect of processing. For example, one layer is specialized for relaying signals from the S-cones and may be part of a"primordial" blue-yellow pathway (Hendry and Reid, 2000). The organization of the retinal inputs to the LGNs, diagrammed in FIGURE 3.15, provides some important insights into how our visual world is mapped to the brain. First, the left LGN receives projections from the left side of the retina in both eyes, and the right LGN receives projections from the right side of both retinas. Second, Parvocellular layers FIGURE 3.14 The lateral geniculate nucleus (LGN) cross section shows the six-layered structure of a primate LGN. Layers 1 and 2 are the magnocellular layers; these larger neurons receive input from the Mganglion cells of the retina. Layers 3-6 are the parvocellular layers, whose smaller cells receive input from the retina's P ganglion cells. The magno- and parvocellular layers are separated by dustlike koniocellular cells. There si an Koniocellular layers Magnocellular layers LGN in each of the brain's two hemispheres.  70 Chapter 3 Spatial Vision: From Spo ts to Stripes Left visual field Right visual field each layer ofthe LGN receives input from one or the other eye. From botom ot A B C D F top, layers ,1,4and 6ofhte right G L N receive input from hte left (contralateral) eye, while layers 2,3, and 5get their input from the right (psilateral eye. Thus, information from the two eyes is segregated ni different layers ni the LGN. Each L G N layer contains ahighly organized map of acomplete half ofthe visual field. Figure 31.5 shows schematically how objectsni the right visual field objects ot hte right of where our gaze si fixated) are mapped onto the different layers fo the left IGN (the right side of the world falls on the left side of hte retina, whose ganglion cells project ot the left LGN). This ordered mapping of the world onto Left eye Right eye the visual nervous system, known as topographical mapping, provides us with a ( e wil return to this point neural basis for knowing where things are ni space w a little later). LGN neurons have concentric receptive fields that are very similar to those A B of retinal ganglion cells: they respond wel to spots and gratings. Given that the FE DIC LGN cells respond to the same patterns as the ganglion cells that provide their input, you might wonder why the visual system bothers with the LGN. Why don't the ganglion cell axons simply travel directly back to the cerebral cortex? One important reason is that the LGN is not merely a stop on the line from retina ot cortex. There are many connections between other parts of the brain and the LGN (Babadi et al., 2010; Dubin and Cleland, 1977). Moreover, there are more feedback connections from the visual cortex to the LGN than feed-forward connections from the LGN to the cortex. Indeed, a recent brain imaging study Left LGN Right LGN (Poltoratski et al., 2019) shows that BOLD activity in human LGN can be mod- F E D C B ified by figure ground organization, even when the figure is presented to one F E D5 С В A5 eye and the background to the other eye. Since information from the two eyes is segregated in different layers ni the LGN, this BOLD modulation must be a F E D С В A4 result of feedback from the cortex. F E D 3 С В Аз It seems that the LGN is a location where various parts of the brain can mod- С ВA ulate input from the eyes. For example, the LGN is part of a larger brain structure D2 2 F E D C B A called the thalamus (the medial geniculate nucleus, part of the auditory pathway, is another portion of the thalamus; see Section 9.3). When you go to sleep, the FIGURE 3.15 Topographic mapping of entire thalamus is inhibited by circuitry elsewhere in the brain that works to keep t w o eyes into the brain you asleep. Thus, even if your eyelids were open while you were sleeping at night, case the letters ABCDEF) from the right you would not see anything in your dimly lit room. Input would travel from your visual field is mapped in an orderly fashion retinas to your LGNs, but the neural signals would stop there before reaching the onto the different layers of the left lateral geniculate nucleus (LGN), and input from the cortex, so they would never be registered. The thalamic inhibition is not complete, left visual field is mapped to the right LGN which si why loud noises (e.g., the alarm clock) or bright lights will be perceived, Information from the two eyes is segregated waking you up. into separate layers. 3.4 The Striate Cortex contralateral Referring to the oppo- site side of the body or brain. fI you place one hand at the back of your head, about an inch or two above the ipsilateral Referring to the same side top of your neck, you should be able to feel a small bump known as the inion. The of the body or brain. receiving area for LGN inputs in the cerebral cortex lies below the inion. This area topographical mapping The orderly has several names: primary visual cortex (V1), area 17, and striate cortex. Striate map ping of the world n i the lateral means "striped"' for the striped pattern V 1 develops following acertain type of stain- genicu late nucleu s and the visual cortex. ing procedure. B y now your'e probably getting hte idea that layers aer na important primary visual cortex (V1), area 17, or property of neural structures ni the visual pathway. The striate cortex consists of The area of the cere- striate cortex bral cortex of the brain that receives six major layers, some of which have sublayers (FIGURE 3.16). Fibers from the LGN direct input s from the lateral geniculate project mainly (but not exclusively) ot layer 4C, with magnocellular axons coming nucleus, as well as feed bac k from other into the upper part of layer C 4 (known as 4Ca) and parvocellular axons projecting b r a i n areas. to the lower part of layer 4C (known as 4Cß) (Yabuta and Callaway, 1998).  3.4 The Striate Cortex 71 FIGURE 3.16 Striate cortex Like the lateral geniculate nucleus (LGN), striate cortex consists of xsi layers. Fibers from the LGN project mainly (but not exclu- sively) to layer 4C. 2/3 4A 4B 4C 5 6 1m m Like the LGN, the striate cortex has a systematic topographical mapping of the visual field. But the striate cortex is not simply a larger version of the LGN. A major and complex transformation of visual information takes place in the striate cortex. For starters, striate cortex contains on the order of 200 million cells-more than 100 times as many as the LGN has! This massive expansion of the number of neurons in Vl may be important for representing our complex natural visual world. FIGURE 3.17 illustrates two important features of the visual cortex: topography and magnification. First, the fact that the image of the woman's right eyebrow (her right; it appears on the left in Figure 3.17) si mapped onto regions corresponding to the numbers 3 and 4 in the striate cortex tells the visual system that the eyebrow must be in positions 3 and 4 of the visual field. This si topographical mapping. Second, information is dramatically scaled from different parts of the visual field. In s imaged on Figure 3.17, the fovea is represented by number 5 on the retina. Object or near hte fovea are processed by neurons ni a large part of the striate cortex, but atiny portion objects imaged ni the far-right or - left periphery are allocated onlycortex of the striate cortex. This distortion of the visual-field map on the is known tion of the fovea is greatly as cortical magnification because the cortical representa magnified compared with the cortical representation of peripheral vision. arms o gain a sense of the extent of this cortical magnification factor, hold your T out ni front of you, put up your index fingers, hold them about 10 cm 4( inches) r. In this position, your right apart, close your left eye, and fixate on your right finge si covering the same amount of visual angle but si falling 10 degrees ot the left of the fovea, si being processed by only 1.5 mm of cortex. The Topography of the Human Cortex ofcorical aera istaly spectedni m-li il. ar) gretdioet visualie regoin m were based on correlating visual-field defects with cortical lesions. That is, someone of the visual with damage to this part of visual cortex would be blind in that part field. FIGURE 3.18A illustrates how eccentricity (distance from the fovea, position 5 27 Chapetr 3 Spatial Vision: From Spots to Stripes FGI URE 31.7 The mapThispiim ngageof objects ni space onto the visual cortex illustrate th top cal mapping (represented yb the positionss bo ographi- of numerals 19-) nad the dramatic magnification fo hte foveal representation ni the cortex. nI the striate cortex, representation of the fper oveaiph(position )Ssi greatly magnified compared ot thatfo eral vision. ni Figure 3.18A) maps onto visual cortex, sa deduced thersteudy ofrsom hainveg been lesions. However,ni hte past02yearsro major advances ni our kn of human visual coretx— ni large part throughowdleedvgeel- opments ni brain-imaging techniques. fI youv' e ever injured your knee or back, you may eb familiar whti 2 ٥ ٦ magnetic resonance imaging. MRI si very useful for anatomical imaging of soft tissues, including hte brain. MRI lets us see the structure of the brain. Func- tional magnetic resonance imaging si a noninvasive technique for measuring and localizing brain activity. As discussed in Section 1.3, fMRI does not measure neural activity directly. Rather, ti measures changes ni blood oxygen level that reflect neural activity. Bolod oxygen level-dependent (BOLD) signals reflect arange of metabolically demanding neural signals (Wandell Left Right and Winawer, 2011). If you compare blood flow when Fove al. eye eye visual stimuli are presented in one portion of the visual image Optic field ot blood flow when the field si blank, you can find chiasm portions of the brain that respond specifically to that Lateral Optic nerve stimulation of that portion of the field. In this way, ti geniculate nucleus si possible to map the topography of V1 ni the living (LGN) Optic human brain (FIGURE 3.18B). Different parts of the Superior tract visual field are mapped onto Figure 3.18 with a color colliculus code. Notice that the red area is just the central few degrees. The blue areas cover vast parts of the more peripheral field, from 20 to 40 degrees. Because of cortical magnification of the central field, these red and blue regions are similar in size. Some Perceptual Consequenc es of Cortical Magnification Optic' Visual acuity declines in an orderly fashion with ec- radiations centricity (Levi, Klein, and Aitsebaomo, 1985), aphe- nomenon demonstrated by Hermann Rudolf Aubert wel over a century ago (Aubert, 1886). FIGURE 31.9 Striate cortex allows you ot demonstrate this phenomenon yourself; the letters are scaled in size such that each one covers an approximately equal cortical area. Why si the foveal representation in the cortex so highly magnified? The visual system must m ake a trade-off. High resolution requires a great number of resources: a dense array of photoreceptors, one-to-one lines from photoreceptors to retinal ganglion cells, and a large chunk of striate cortex (not to mention the real estate in other areas of cortex necessary ot do something with the visual information coming out fo V1). T o se the entire visual field with such high resolution, w e might need eyes  3.5 Receptive Fields in Striate Cortex 73 (A) (B) Calcarine sulcus -10. 255 Fovea 4 0 5 10 20 2.5 5 10 20 4 0 1 cm Eccentricity (degrees) FIGURE 3.18 Cortical mapping Mapping the visual field onto shown in (B). The red area is just the central few degrees adjacent the cortex as deduced from studying lesions (A) and from func- to the fovea, while the blue areas cover substantially more of the tional magnetic resonance imaging (B). Different parts of the visual peripheral field, from 20 to 40 degrees. field are mapped onto the cortex according to the color code FIGURE 3.19 A cortically s c a l e d letter chart A letter chart in which the letter size increases with eccentricity in proportion to the inverse cortical magnification factor. fI you fixate your gaze on the far-left side of the figure, all seven letters should be equally easy to see because those on the right, which are in the periphery, are so much larger. and brains too large to fit in our heads! Thus, we have evolved a visual system that provides high resolution in the center and lower resolution in the periphery. If you need to process the details of an object that appears in the corner of your eye, you can simply turn your eye or head so that the object falls on the fovea instead. FURTHER DISCUSSION of eye movements can be found in Section 8.4. 3.5 Receptive Fields in Striate Cortex In 1958, David Hubel and Torsten Wiesel began work as postdoctoral students in Stephen Kuffler's laboratory. Their goal was to extend Kuffler's groundbreaking work on retinal ganglion cells and to apply it to the cortex. So they began trying to map the receptive fields of neurons in striate cortex of cats using spots of light, much as Kuffler (1953) had done earlier (see Section 2.4). Recall from Chapter 2that the receptive field of a neuron si the region in space in which the presence of a stimulus alters the neuron's firing rate. To Hubel and Wiesel's dismay, they found that a cat's cortical cells hardly responded at all to the same spots that made its ganglion cells fire like crazy. To project their stimuli onto 47 Chapter 3 Spatial Vision: From Spots to Stripes orientation t u n i n g neurons in striate cortexT eh tendency of hte retina, Hubel and Wiesel inserted a glass slide with a black spot into aslot ni ym to resp ond lilaopt to certain orien tatio ns and aspecial ophthalmoscope (that's hte instrument the doctor uses when she shines less ot others. abright light into your eye ot see your retina). One day, h tey had been recording ocular dominance The property of from aneuron without much luck, when suddenly the cell emitted astrong burst hte receptive fields of striate cortex of firing as they inserted hte glass slide into the slot. Eventually, they realized that neurons by which they demo nstra te a the response had nothing ot do with the spot itself; instead, the cell had been preference, responding somewhat more responding ot the shadow cast by the edge of the glass slide as ti swept across the rapidly when a stimulus is presented ni one eye than when ti is presented ni the ophthalmoscope's light path. And the rest, as they say, si history. Hubel related other. this story when eh and Wiesel received the 1981 Nobel Prize ni Physiology or simple cell A cortical neuron whose Medicine for uncovering many of the remarkable properties of the visual cortex receptive field has clearly defined excit- (Hubel, 1982) (FIGURE 3.20A). atory and inhibitory regions. Hubel and Wiesel's most fundamental discovery was that the receptive fields of striate cortex neurons are not circular, as they are in the retina and LGN. Rather, they are elongated. As a result, they respond much more vigorously to bars, lines, edges, and gratings than to round spots of light. Orientation Selectivity Further investigation by Hubel and Wiesel (1962) uncovered a number of other important properties of the receptive fields of neurons in striate cortex. First, an individual neuron wil not respond equivalently ot just any old stripe ni its receptive field. tI responds best when the line or edge is at just the (A) right orientation and hardly at all when the line si tilted more than 30 degrees away from the optimal orientation (a change equivalent to movement of the minute hand of a clock from 12 to 1). Scientists call this selective respon- siveness orientation tuning: the cell is tuned to detect lines in a specific orientation. Atypical orientation tuning function looks like the plot in FIGURE 3.20B. The neuron featured here fires vigorously when the line is oriented vertically, but the response tapers off rapidly as the line is tilted one way or another, diminishing to close to the cell's resting rate when the line is tilted about 30 degrees in either direction. Other cells in striate cortex are selective for horizontal lines and lines at 45 degrees, 20 degrees, 62 degrees, and so on, so the population of neurons as a whole detects all possible orientations. However, more cells are responsive (B) to horizontal and vertical orientations than to obliques (De Valois, Yund, and Hepler, 1982; B. L,i Peterson, and Free- man, 2003). This physiological finding meshes wel whti the psychophysical finding that humans have somewhat lower visual acuity and contrast sensitivity for oblique Firing rate targets than for horizontal and vertical targets. FIGURE 3.20 Nobel men A ( ) David Hubel (left) and Tor- sten Wiesel, shown here ni their lab, received the 1981 Nobel Prize for discoveries that furthered understanding of informa- tion processing ni the visual system. (B) Orientation tuning function of a cortical cell. The neuron fires vigorously w hen the line si oriented vertically, but hardly at all when the line o-ri entation si changed by 03 degrees. While this example si for Orientation of line (degrees) acell tuned ot vertical lines, other cells are tuned ot different ori ent atio ns.  3.5 Receptive Fields ni Striate Cortex 75 How are the circular receptive fields in the LGN transformed into the elongated receptive fields in striate cortex? Hub Wiesel, 1979) suggested a very simple sche el and Wiesel (Hubel and me to explain this transformation. Simply put, their idea was that the concent ric LGN cells that feed into a cortical cell are all in a row (FIGURE 3.21 ). Later studies (e.g.,.J S. Anderson et al., 2000) have shown that the arrange ment of LGN inputs for establishing the orientation selectivity of striate cortex is indeed crucial other evidence suggests that neural inte cells. However, ractions (e.g., lateral inhibition; see Section 2.4) within the cortex also play an important role in the dynamics of orientation tuning (Pugh et al., 2000). LGN Other Receptive-Field Properties cells Cortical cells don't just respond to bars, lines , and edges. Like retinal gangli- on cells, they also respond well to gratings (which are, after all, collections of Striate lines). And, like ganglion cells, they respond best to gratings that have just Striate (V1) the receptive field cortex cell right spatial frequency to fill the receptive-field center. That is, each striate cortex cell is tuned to a particular spatial frequency, which corres ponds to a particular line width. Indeed, cortical cells are much more narrowly tuned FIGURE 3.21 How striate cortex cells get (that