What the Frog's Eye Tells the Frog's Brain PDF

6 WHAT THE FROG'S EYE T E L L S THE FROG'S BRAIN J.Y. Lettvin*, H.R. Maturana**, W.S. McCulloch*** and W.H. P i t t s « * Strmmary—In this paper, we analyse the activity of single fibres in the optic nerve of...

6 WHAT THE FROG'S EYE T E L L S THE FROG'S BRAIN J.Y. Lettvin*, H.R. Maturana**, W.S. McCulloch*** and W.H. P i t t s « * Strmmary—In this paper, we analyse the activity of single fibres in the optic nerve of a frog. Our method is to find what sort of stimulus causes the largest activity in one nerve fibre and then what is the exciting aspect of that stimulus such that variations in everything else cause little change in the response. It has been known for the past 20 years that each fibre is connected not to a few rods and cones in the retina but to very many over a fair area. Our results show that for the most part within that area, it is not the light intensity itself but rather the pattern of local variation of intensity that is the exciting factor. There are four types of fibres, each type concerned with a different sort of pattern. Each type is uniformly distributed over the whole retina of the frog. Thus, there are four distinct parallel distributed chan- nels whereby the frog's eye informs his brain about the visual image in terms of local pattern independent of average illumi- nation. We describe the patterns and show the functional and anatomical separation of the channels. This work has been done on the frog, and our interpretation applies only to the frog. INTRODUCTION Behaviour of a Frog A frog hunts on land by vision. He escapes enemies mainly by seeing them. His eyes do not move, as do ours, to follow prey, attend suspicious events, or search for things of interest. If his body changes its position with respect to gravity or the whole R e s. Lab. of Electronics and Dept. of Biology, Mass. Inst. Tech., Cambridge, Mass. **Res. Lab. of Electronics, Mass. Inst. Tech., Cambridge, Mass., on leave from the University of Chile, Santiago, Chile. ***Res. Lab. of Electronics, Mass. Inst. Tech., Cambridge, Mass. 95  KEY PAPERS visual world is rotated about him, then he shows compensatory eye movements. These movements enter his hunting and evading habits only, e.g., as he s i t s on a rocking lily pad. Thus his eyes are actively stabilized. He has no fovea, or region of greatest acuity in vision, upon which he must centre a part of the image. He also has only a single visual system, retina to colliculus, not a double one such as ours where the retina sends fibres not only to colliculus but to the lateral geniculate body which relays to cerebral cortex. Thus, we chose to work on the frog because of the uniformity of his retina, the normal lack of eye and head movements except for those which stabilize the retinal image, and the relative simplicity of the connection of his eye to his brain. The frog does not seem to see or, at any rate, is not concerned with the detail of stationary parts of the world around him. He will starve to death surrounded by food if it is not moving. His choice of food is determined only by size and movement. He will leap to capture any object the size of an insect or worm, providing it moves like one. He can be fooled easily not only by a bit of dangled meat but by any moving small object. His sex life is conducted by sound and touch. His choice of paths in escaping enemies does not seem to be governed by anything more devious than leaping to where it is darker. Since he is equally at home in water and on land, why should it matter where he lights after jumping or what particular direction he takes? He does remember a moving thing providing it stays within his field of vision and he is not distracted. Anatomy of Frog Visual Apparatus The retina of a frog is shown in Figure la. Between the rods and cones of the retina and the ganglion cells, whose axons form the optic nerve, lies a layer of connecting neurons (bipolars, horizontals, and amacrines). In the frog there are about 1 million receptors, 2% to 3% million connecting neurons, and half a million ganglion cells (Maturana, 1959). The connections are such that there is a synaptic path from a rod or cone to a great many ganglion cells, and a ganglion cell receives paths from a great many thousand receptors. Clearly, such an arrangement would not allow for good resolution were the retina meant to map an image in terms of light intensity point by point into a distribu- tion of excitement in the optic nerve. There i s only one layer of ganglion cells in the frog. These cells are half a million in number (as against one million rods and cones). The neurons are packed together tightly in a sheet 96  BRAIN PHYSIOLOGY AND PSYCHOLOGY Figure 1(a). This i s a diagram of the frog retina done by Ramon y Cajal over 50 years ago (Cajal, 1909-11). The rods and cones are the group of elements in the upper left quarter of the picture. T o their bushy bottom ends are connected the bipolar c e l l s of the intermediate layer, for example, f, g, and h. Lateral connecting neurons, called horizontal and amacrine cells, also occur in this layer, for example, i, j and m. The bipolars send their axons down to arborize in the inner plexiform layer, roughly the region bounded by cell m above and the bodies of the ganglion cells, o, p and q, below. In this sketch, Ramon has the axons of the bipolar c e l l s emitting bushes at all l e v e l s in the plexiform layer; in fact, many of them branch at only one or two levels. Compare the dendrites of the different ganglion cells. Not only do they spread out at different l e v e l s in the plexiform layer, but the patterns of branching are different. Other ganglion cells, not shown here, have multiple arbors spreading out like a plane tree at two or three levels. If the terminals of the bipolar c e l l s are systematically arranged in depth, making a laminar operational map of the rods and cones in terms of very local contrast, colour, ON, O F F , etc., then the different shapes of the ganglion c e l l s would correspond to different combinations of the local operations done by the bipolars. Thus would arise the more complex operations of the ganglion c e l l s as described in the text (Continued) 4* 97  KEY PAPERS Figure 1(b). This is Ramon y Cajal's diagram of the total decussation or crossing of the optic nerve fibres in the frog (Cajal, 1909-11). He made this picture to explain the value of the crossing as preserving continuity in the map of the visual world. 0 is the optic nerve and C is the superior colliculus or optic tectum (the names are synonymous) (Continued) 98  Tr. apt. Figure 1(c). This i s Ariens-Kapper's picture of the cross section of the brain of a frog through the colliculus, which i s the upper or dorsal part above the enclosed space (Continued) Figure 1(d). This i s Pedro Ramon C a j a l ' s diagram of the nervous organization of the tectum of a frog. The terminal bushes of the optic nerve fibres are labelled a, b, and c. A, B, C, D and E are tectal c e l l s receiving from the optic nerve fibres. Note that the axons of these c e l l s come off the dendrites in stratum 7, which we call the palisade layer. The endings discussed in this paper lie between the surface and that stratum 99  KEY PAPERS at the level of the cell bodies. Their dendrites, which may extend laterally from 50 to 500 microns, interlace widely into what is called the inner plexiform layer, which is a close-packed neuropil containing the terminal arbors of those neurons that lie between receptors and ganglion cells. Thus, the amount of overlap of adjacent ganglion cells is enormous in respect to what they see. Morphologically, there are several types of these cells that are as distinct in their dendritic patterns as different species of trees, from which we infer that they work in different ways. The anatomy shown in the figures is that found in standard references. Further discussion of anatomical questions and additional origi- nal work on them will appear in a later publication. Physiology as Known up to This Study Hartline (1938) first used the term receptive field for the region of retina within which a local change of brightness would cause the ganglion cell he was observing to discharge. Such a region is sometimes surrounded by an annulus, within which changes of brightness affect the cell's response to what is occurring in the receptive field, although the cell does not discharge to any event occurring in the annulus alone. Like Kuffler (1953), we consider the receptive field and its interacting annulus as a single entity, with apologies to Dr. Hartline for the slight change in meaning. Hartline found three sorts of receptive field in the frog: ON, ON-OFF, and OFF. If a small spot of light suddenly appears in the receptive field of an ON-cell, the discharge soon begins, increases in rate to some limit determined by the intensity and area of the spot, and thereafter slowly declines. Turning off the spot abolishes the discharge. If the small spot of light suddenly appears or disappears within the field of an ON-OFF cell, the discharge is short and occurs in both cases. If the spot of light disappears from the field of an OFF cell, the discharge begins immediately, decreases slowly in frequency, and lasts a long time. It can be abolished promptly by turning the spot of light on again. For all three sorts of field, sensitivity is greatest at the centre of each field and least at the periphery. Barlow (1953) extended Hartline's observations. He observed that the OFF cells have an adding receptive field, i.e., the re- sponse occurs always to OFF at both centre and periphery of that field, and that the effect of removing light from the periphery adds to the effect of a reduction of light at the centre, with a weight decreasing with distance. 100  BRAIN P H Y S I O L O G Y AND P S Y C H O L O G Y The ON-OFF cells, however, have differencing receptive fields. A discharge of several spikes to the appearance of light in the centre is much diminished if a light is turned on in the extreme periphery. The same interaction occurs when these lights are re- moved. Thus, an ON-OFF cell seems to be measuring inequality of illumination within its receptive field. (Kuffler (1953) at the same time showed a similar mutual antagonism between centre and periphery in each receptive field of ganglion cells in the eye of a cat, and later Barlow, Kuffler and Fitzhugh (1957) showed that the size of the c a t ' s receptive fields varied with general illumination). Barlow saw that ON-OFF cells were profoundly sensitive to movement within the receptive field. The ON cells have not been characterized by similar methods. These findings of Hartline and Barlow establish that optic nerve fibres (the axons of the ganglion cells) do not transmit information only about light intensity at single points in the retina. Rather, each fibre measures a certain feature of the whole distribution of light in an area of the receptive field. There are three sorts of function, or areal operation, according to these authors, so that the optic nerve looks at the image on the retina through three distributed channels. In any one channel, the over- lap of individual receptive fields is very great. Thus one is led to the notion that what comes to the brain of a frog is this: for any visual event, the OFF channel tells how much dimming of light has occurred and where; the ON-OFF channel tells where the boundaries of lighted areas are moving, or where local in- equalities of illumination are forming; the ON channel shows (with a delay) where brightening has occurred. To an unchanging visual pattern, the nerve ought to become fairly silent after a while. Consider the retinal image as it appears in each of the three distributed channels. For both the OFF and ON channels, we can treat the operation on the image by supposing that every point on the retina gives rise to a blur about the size of a receptive field. Then the OFF channel tells, with a long decay tirrie, where the blurred image is darkened, and the ON channel tells with a delay and long decay where it is brightened. The third channel, ON- OFF, principally registers moving edges. Having the mental picture of an image as it appears through the three kinds of chan- nel, we are still faced with the question of how the animal abstracts what is useful to him from his surroundings. At this point, a safe position would be that a fair amount of data reduction has in fact been accomplished by the retina and that the interpretation is the work of the brain, a yet-to-be unravelled mystery. Yet the nagging 101  KEY P A P E R S worries remain: why are there two complementary projections of equally poor resolution? Why is the mosaic of receptors so use- lessly fine? Initial Argument The assumption has always been that the eye mainly s e n s e s light, whose local distribution is transmitted to the brain in a kind of copy by a mosaic of impulses. Suppose we held otherwise, that the nervous apparatus in the eye is itself devoted to detect- ing certain patterns of light and their changes, corresponding to particular relations in the visible world. If this should be the case, the laws found by using small spots of light on the retina may be true and yet, in a sense, be misleading. Consider, for example, a bright spot appearing in a receptive field. Its actual and sensible properties include not only intensity, but the shape of its edge, its size, curvature, contrast, etc. We decided then how we ought to work. First, we should find a way of recording from single myelinated and unmyelinated fibres in the intact optic nerve. Second, we should present the frog with as wide a range of visible stimuli as we could, not only spots of light but things he would be disposed to eat, other things from which he would flee, sundry geometrical figures, stationary and moving about, etc. From the variety of stimuli we should then try to discover what common features were abstracted by whatever groups of fibres we could find in the optic nerve. Third, we should seek the anatomical basis for the grouping. This programme had started once before in our laboratory with A.M. Andrew (1955a,b) of Glasgow who unfortunately had to return to Scotland before the work got well under way. However, he had reported in 1957 that he found elements in the colliculus of the frog that were sensitive to movement of a spot of light (a dot on an oscilloscope screen) even when the intensity of the spot was so low that turning it on and off produced no response. In particular, the elements he observed showed firing upon movement away from the centres of their receptive fields, but not to centripetal movements. As will appear later, this sort of response i s a natural property of OFF fibres. (ACTUAL) METHODS Using a variant of Dowben and R o s e ' s platinum black-tipped electrode described in another paper of this issue, we then began a systematic study of fibres in the optic nerve. One of the authors (H.R.M.) had completed the electron microscopy of optic nerve in frogs (Maturana, 1958), and with his findings we were able to 102  BRAIN P H Y S I O L O G Y AND PSYCHOLOGY understand quickly why certain kinds of record occurred. He had found that the optic nerve of a frog contains about half a million fibres (ten times the earlier estimates by light microscopy). There are 30 times as many unmyelinated axons as myelinated, and both kinds are uniformly distributed throughout the nerve. The axons lie in small densely packed bundles of five to 100 fibres with about 100 X between axons, each bundle surrounded by one or more glial cells (Maturana, 1958). But along the nerve no bundle maintains its identity long, for the component fibres so braid be- tween bundles that no two fibres stay adjacent. Thus the place a fibre crosses one section of the nerve bears little relation to its origin in the retina and little relation to where it crosses another section some distance away. Fibres are so densely packed that one might suppose such braiding necessary to prevent serious interactions. On the other hand, the density makes the recording easier. A glial wall sur- rounds groups rather than single fibres, and penetration of the wall brings the tip among really bare axons each surrounded by neighbours whose effect is to increase the external impedance to its action currents, augmenting the external potential in propor- tion. Thus, while we prefer to use platinum black tips to improve the ratio of signal to noise, we recorded much the same popula- tion with ordinary sharp micro-electrodes of bright Pt or Ag. The method records equally well from unmyelinated and myelinated fibres. We used Rana pipiens in these experiments. We opened a small flap of bone either just behind the eye to expose the optic nerve, or over the brain to expose the superior colliculus. No further surgery was done except to open the membranes of connective tissue overlying the nervous structure. The frog was held in extension to a cork platform and covered with moist cloth. An animal in such a position, having most of his body surface in physical contact with something, goes into a still reaction—i.e., he will not even attempt to move save to pain, and except for the quick small incision of the skin at the start of the operation our procedure seems to be painless to him. With the animal mounted, we confront his eye with an aluminium hemisphere, 20 mils thick and 14 inches in diameter, silvered to a matte grey finish on the inner surface and held concentric to the eye. On the inner surface of this hemisphere, various objects attached to small magnets can be moved about by a large magnet moved by hand on the outer surface. On our hemisphere, 1 degree is slightly less than an eighth of an inch long. In the tests illustrated, we use as stimu- lating objects a dull black disk almost 1 degree in diameter and 103  KEY PAPERS a rectangle 30 degrees long and 12 degrees wide. However, in the textual report, we use a variety of other objects. As an indi- cator for the stimulus, we first used a phototube looking at an image of the hemisphere taken through a camera lens and focus- sed on the plane of a diaphragm. (Later we used a photomulti- plier, so connected as to give us a logarithmic response over about four decades.) Thus we could vary how much of the hemi- sphere was seen by the stimulus detector and match that area in position and size against the receptive field of the fibre we were studying. The output of this arrangement is the stimulus line in the figures. FINDINGS There are four separate operations on the image in the frog's eye. Each has its result transmitted by a particular group of fibres, uniformly distributed across the retina, and they are all nearly independent of the general illumination. The operations are: (1) sustained contrast detection; (2) net convexity detection; (3) moving edge detection; and (4) net dimming detection. The first two are reported by unmyelinated fibres, the last two by myelinated fibres. Since we are now dealing with events rather than point excitations as stimuli, receptive fields can only be defined approximately, and one cannot necessarily distinguish concentric subdivisions. The fibres reporting the different oper- ations differ systematically not only in fibre diameter (or conduc- tion velocity) but also in rough size of receptive field, which ranges from about 2 degrees diameter for the first operation, to about 15 degrees for the last. The following description of these groups is definite. (1) Sustained Contrast Detectors An unmyelinated axon of this group does not respond when the general illumination is turned on or off. If the sharp edge of an object either lighter or darker than the background moves into its field and stops, it discharges promptly and continues discharging, no matter what the shape of the edge or whether the object is smaller or larger than the receptive field. The sustained discharge can be interrupted (or greatly reduced) in these axons by switch- ing all light off. When the light is then restored, the sustained discharge begins again after a pause. Indeed the response to turning on a distribution of light furnished with sharp contrast within the field is exactly that reported by Hartline for his ON fibres. In some fibres of this group, a contrast previously within 104  BRAIN PHYSIOLOGY AND PSYCHOLOGY -vjliij 111. (b). "\ff| 1 1 i \ s (c) JlLtìff^ L- L_ (e) Figure 2. Operation (1)—contrast detectors. The records were all taken directly with a Polaroid camera. The spikes are clipped at the lower end just above the noise and brightened on the screen. Occasional spikes have been intensified by hand for purposes of reproduction. The resolution is not good but we think that the responses are not ambiguous. Our alternate recording method is by means of a device which displays the logarithm of pulse interval of signals through a pulse height pick-off. However, such records would take too much explanation and would not add much to the substance of the present paper, (a) This record is from a single fibre in the optic nerve. The base line is the output of a photocell watching a somewhat larger area than the receptive field of the fibre. Darkening is given by downward deflection. The response is seen with the noise clipped off. The fibre discharge to movement of the edge of a 3 degree black disk passed in one direction but not to the reverse movement (time marks, 20 per second), (b) The same fibre shown here giving a continued response when the edge stops in the field. The response disappears if the illumination is turned off and reappears when it is turned on. Below is shown again the asymmetry of the response to a faster movement (time marks, 20 per second), ( c ) The same fibre is stimulated here to show asymmetrical response to the 3 degree black object moved in one direction, then the reverse and the stimuli are repeated under a little less than a 3-decade change of illumination in two steps. The bottom record is in extremely dim light, the top in very bright light (time marks, 20 per second), (d) In the bottom line, a group of endings from such fibres is shown recorded from the first layer in the tectum. A black disk 1 degree in diameter is moved first through the field and then into the field and stopped. In the top line, the receptive field is watched by a photomultiplier (see text) and darkening is given by upward deflection (time marks, 5 per second for all tectal records), ( e ) OFF and ON of general illumination has no effect on these fibres. (/) A 3 degree black disk is moved into the field and stopped. The response continues until the lights are turned OFF but reappears when the lights are turned ON. These fibres are non- erasable. (Continued) 105  KEY PAPERS (J) i juLj.| iBj Figure 2. (g) A very large black square is moved into the field and stopped. The response to the edge continues so long as the edge is in the field. (A) T h e 3 degree disk is again moved into the field and stopped. When it leaves, there i s a slight after-discharge, (/) A 1 degree object i s moved into the field, stopped, the light i s then turned off, then on, and the response comes back. The light is, however, a little l e s s than 3 0 0 x dimmer than in the next frame. Full ON and O F F are given on the rectangular calibration on the right. (/) The same procedure as in ( i ) is done under very bright light. The return of response after reintroducing the light seems more prolonged—but this is due only to the fact that, in (/), the edge was not stopped in optimal position the field is 'remembered' after the light is turned off, for they will keep up a low mutter of activity that is not present if no contrast was there before. That this is not an extraordinary sensi- tivity of such an element in almost complete darkness can be shown by importing a contrast into its receptive field after dark- ening in the absence of contrast. No mutter occurs then. This memory lasts for at least a minute of darkness in some units. In Figure 2 we see the response of such a fibre in the optic nerve. We compare these responses with full illumination (a 60-watt bulb and reflector mounted a foot away from the plane of the opening of the hemisphere) to those with less than 1/300 as much light (we put a variable resistance in series with the bulb so that the colour changed also). We are struck by the smallness of the resulting change. In very dim light where we can barely see the stimulating object ourselves, we still get very much the same response. (2) Net Convexity Detectors These fibres form the other subdivision of the unmyelinated population, and require a number of conditions to specify when 106  BRAIN P H Y S I O L O G Y AND P S Y C H O L O G Y they will respond. To our minds, this group contains the most remarkable elements in the optic nerve. Such a fibre does not respond to change in general illumination. It does respond to a small object (3 degrees or less) passed through the field; the response does not outlast the passage. It continues responding for a long time if the object is imported and left in the field, but the discharge is permanently turned off (erased) by a transient general darkness lasting 1/10 second or longer. We have not tried shorter obscurations. The fibre will not respond to the straight edge of a dark object moving through its receptive field or brought there and stopped. If the field is about 7 degrees in diameter, then, if we move a dark square 8 degrees on the side through it with the edge in advance there is no response, but if the corner is in advance then there is a good one. Usually a fibre will respond indefinitely only to objects which have moved into the field and then lie wholly or almost wholly interior to the receptive field. The discharge is greater the greater the convexity, or positive curvature, of the boundary of the dark object until the object becomes as small as about xh the width of the receptive field. At this point, we get the largest response on moving across that field, and a good, sus- tained response on entering it and stopping. As one uses smaller and smaller objects, the response to moving across the field begins to diminish at a size of about 1 degree, although the sus- tained response to coming in and stopping remains. In this way we find the smallest object to which these fibres respond is l e s s than 3 minutes of arc. A smooth motion across the receptive field has l e s s effect than a jerky one, if the jerks recur at intervals longer than % second. A displacement barely visible to the ex- perimenter produces a marked increase in response which dies down slowly. Any checked or dotted pattern (in the latter case, with dots no further apart than half the width of the receptive field) moved as a whole across the receptive field produces little if any response. However, if any dot within the receptive field moves differentially with respect to the background pattern, the response is to that dot as if it were moving alone. A group of two or three distinct spots enclosed within the receptive field and moved as a whole produce l e s s direct response to movement and much l e s s sustain- ed response on stopping than if the spots are coalesced to a single larger spot. A delightful exhibit u s e s a large colour photograph of the natural habitat of a frog from a frog's eye view, flowers and grass. We can move this photograph through the receptive field of such a 107  KEY PAPERS fibre, waving it around at a 7-inch distance: there is no response. If we perch with a magnet a fly-sized object 1 degree large on the part of the picture seen by the receptive field and move only the object we get an excellent response. If the object is fixed to the picture in about the same place and the whole moved about, then there is none. Finally, the response does not depend on how much darker the object is than its background, so long as it is distinguishably so and has a clear-cut edge. If a disk has a very dark centre and merges gradually into the grey of the background at the boundary, the response to it is very much l e s s than to a uniform grey disk only slightly darker than the background. Objects lighter than the background produce almost no response unless they have enough relief to cast a slight shadow at the edge. All the responses we have mentioned are relatively independent of illumination, and Figure 3 taken as described in the caption shows the reactions to a 3 degree object and the large rectangle under some of the conditions described. General Comments on Groups (1) and (2) The two sorts of detectors mentioned seem to include all the unmyelinated fibres, with conduction velocities of 20 to 50 cm. The two groups are not entirely distinct. There are transition cases. On one hand, some convexity detectors respond well to very slightly curved edges, even so far as to show an occasional sustained response if that edge is left in the field. They may also not be completely erasable (though very markedly affected by an interruption of light) for small objects. On the other hand, others of the same group will be difficult to set into an indefi- nitely sustained response with any object, but only show a fairly long discharge, acting thereby more a s detectors of edges although never reacting to straight edges. Nevertheless the distribution of the unmyelinated axons into two groups is very marked. Any fibre of either group may show a directional response—i.e., there will be a direction of movement that may fail to excite the cell. For the contrast fibres, this will also appear as a nonexciting angle of the boundary with respect to the axis of the frog. Such null directions and angles cancel out in the aggregate. (3) Moving-Edge Detectors These fibres are myelinated and conduct at a velocity in the neighbourhood of 2 metres per second. They are the same as Hartline's and Barlow's ON-OFF units. The receptive field is about 12 degrees wide. Such a fibre responds to any distinguish- able edge moving through its receptive field, whether black 108  BRAIN PHYSIOLOGY AND PSYCHOLOGY (a) (b) (c) (d).