Visual Resolution with Retinal Implants from Cat Visual Cortex PDF

Vision Research 46 (2006) 2675–2690 www.elsevier.com/locate/visres Visual resolution with retinal implants estimated from rec...

Vision Research 46 (2006) 2675–2690 www.elsevier.com/locate/visres Visual resolution with retinal implants estimated from recordings in cat visual cortex Reinhard Eckhorn a,¤, Marcus Wilms a, Thomas Schanze a, Marcus Eger a, Lutz Hesse a,b, Ulf T. Eysel c, Zoltán F. Kisvárday c,1, Eberhart Zrenner d, Florian Gekeler d, Helmut Schwahn d, Keisuke Shinoda d, Helmut Sachs e, Peter Walter f a Department of Physics, Neurophysics Group, Philipps-University Marburg, Germany b Eye Clinic, Philipps-University Marburg, Germany c Department of Neurophysiology, Medical Faculty, Ruhr-University Bochum, Germany d University Eye Hospital, Medical Faculty, University of Tübingen, Germany e University Eye Hospital, Medical Faculty, University of Regensburg, Germany f University Eye Hospital, Medical Faculty, University of Aachen, Germany Received 14 May 2005; received in revised form 7 January 2006 Abstract We investigated cortical responses to electrical stimulation of the retina using epi- and sub-retinal electrodes of 20–100 m diameter. Temporal and spatial resolutions were assessed by recordings from the visual cortex with arrays of microelectrodes and optical imaging. The estimated resolutions were »40 ms and »1° of visual angle. This temporal resolution of 25 frames per second and spatial resolution of about 0.8 cm at about 1 m and correspondingly 8 cm at 10 m distance seems suYcient for useful object recognition and visuo-motor behavior in many in- and out-door situations of daily life. © 2006 Elsevier Ltd. All rights reserved. Keywords: Retina prosthesis; Visual cortex; Spatial and temporal resolution; Electrical stimulation 1. Introduction percentage of inner retinal neurons remain histologically intact (Santos et al., 1997). In particular, a large number of Retinal photoreceptors transform visual stimuli into the retinal ganglion cells stay functionally alive and can electrical signals in normal vision. These signals are pro- transmit action potentials via the optic tract. Apart from cessed by intraretinal neural networks and the resulting the substitution of retinal function by replacing deterio- visual information is submitted by retinal ganglion cells to rated outer retinal cells with intact sub-retinal transplants subsequent visual centers. This is not so in the case of dis- (Seiler, Aramant, & Ball, 1999), most current concepts for eases of the outer retina, including macula degeneration restituting retinal functions are based on the electrical stim- and retinitis pigmentosa. They lead to a progressive and ulation of the remaining intact visual neurons. Electrical Wnally total loss of the photoreceptors. However, a large stimuli are meant to coarsely mimic visual inputs to subse- quent visual processing. * Corresponding author. Fax: +49 6421 282 70 34. 1.1. Previous work E-mail addresses: [email protected], reinhard.eckhorn@ physik.uni-marb (R. Eckhorn). 1 Present address: University of Debrecen, Department of Anatomy, In recent years, two main types of retinal implants have Histology and Embryology, Hungary. been developed using either sub-retinal or epi-retinal 0042-6989/$ - see front matter © 2006 Elsevier Ltd. All rights reserved. doi:10.1016/j.visres.2006.01.034 2676 R. Eckhorn et al. / Vision Research 46 (2006) 2675–2690 electrode arrays (e.g., Chow & Chow, 1997; Chow et al., Maynard, Rousche, & Warren, 1999; Rizzo et al., 1999; 2001; Eckmiller, 1997; Eger, Wilms, Eckhorn, & Schanze, Schanze et al., 2002; Stett et al., 2000; Wilms et al., 2003; 2005; Humayun et al., 1996, 1999, 2003; Peyman et al., Wyatt & Rizzo, 1996; Zrenner et al., 1997, 1999; Zrenner, 1998; Rizzo & Wyatt, 1997; Schanze, Wilms, Eger, Hesse, & 2002). However, one crucial question remains regarding the Eckhorn, 2002; Schwahn et al., 2001; Wilms, Eger, Schanze, type of neural responses evoked in the visual cortex by & Eckhorn, 2003; Zrenner et al., 1997, 1999; Zrenner, 2002). stimulation with retinal implants. To be potentially useful The sub-retinal devices are implanted between the pig- for visual perception, spatial, temporal, and intensity reso- ment epithelial layer and the outer layers of the retina. Elec- lutions must be suYcient to discriminate objects in a static trical stimulation with such devices directly activates the environment and to perceive motion in the dynamic scenes overlying retinal tissue, for example the outer plexiform of everyday life. layer and bipolar cells in the case of photoreceptor degener- ation. The stimulation parameters can partly be determined 1.2. Minimal requirements for useful artiWcial vision by the electronics of the implant, and local stimulation is triggered by the light falling onto a micro-photo-diode Object recognition relies mainly on spatial resolution. A (or—transistor) array. Stett, Barth, Weiss, Haemmerle, and resolution better than 0.5° of visual angle corresponds to a Zrenner (2000) have shown in in vitro experiments in nor- visual acuity of 1/35 where clinically vision testing starts. mal and degenerated rat retinae, stimulated by multi-elec- 0.5° visual angle is therefore the ultimate goal in the devel- trode arrays, that spatial resolution at retinal level, as opment of retina implants. 2° resolution will lead to determined by ganglion cell recordings, is at least 70 m improvement of performance in daily life, like the recogni- corresponding to 1/3° visual angle. Chow and colleagues tion of objects that are important for taking meals, for (2001; Chow et al. 2002) have reported implantation of a washing, and dressing. Finally, 10° resolution might be passive sub-retinal device (without external energy supply) suYcient for mobility and orientation, allowing subjects to into the eye of patients, using the method of Peyman (1998). perceive the presence of large objects and their movement This study was designed to examine biocompatibility of directions (e.g., Legge, Ahn, Klitz, & Luebker, 1997). How- their sub-retinal devices but not their function. After a very ever, gaining at least some improvement of performance in short initial phase of diVuse phosphene perception at the daily life would be highly valuable for blind people and site of the device no further light perception was reported even regaining some mobility and orientation could by the patients. However, some improvement of still func- improve their quality of life considerably. In normal visual tioning retinal areas, probably due to the release of growth processing, spatial resolution can be estimated at any stage factors induced by the surgery, the result of subthreshold of the visual system from the size of the classical receptive electrical stimulation or the presence of the device itself, Welds (CRFs) of its neurons. Inversely, spatial resolution have been reported by some of the patients. can also be estimated from the spread of neural activation Epi-retinal implants are placed from the vitreous side in the primary visual cortex in response to focal retinal onto the ganglion cell and nerve Wber layer of the retina. In stimulation as is done with small visual stimuli and electri- response to visual stimulation through an external image cal impulses in this investigation. sensor-array, combined with a real-time retina-processor Minimal requirements for useful artiWcial vision can be (e.g., see Eckmiller, 1997), the epi-retinal implant evokes estimated by psychophysical methods (e.g., Cha, Horch, & action potentials in retinal ganglion cells and their Wbers. Normann, 1992a; Legge et al., 1997; Sommerhalder et al., Finally, both types of implants send visual information in 2003). Near perfect reading at central retinal projection the form of spike patterns of the ganglion cells via the optic sites requires at least 300 sampling points of the text to be nerve to central visual structures. read within about 5° retinal eccentricity. If safe navigation In recent experiments, the retina of blind patients was in dynamic out-door environments is also required, stimu- electrically stimulated and they reported sensations of light lation should include eccentricities of 10–15° (Cha et al., spots in response to focal electrical stimulation of the inner 1992a; Geruschat, Turano, & Stahl, 1998) where spatial res- retina (Humayun et al., 1996, 1999, 2003; Rizzo & Wyatt, olution for navigational purposes can be lower. About 200 1997). These experiments clearly demonstrated the feasibil- additional sampling points are suYcient within this outer ity of generating perception of light patterns in blind people “navigational belt.” The number of required sampling by retinal stimulation. Presently, no Wnal conclusion can be points and their spatial distance deWne the required number drawn from the Wrst experiments with patients about how and spacing of stimulation electrodes in a retinal implant. close retinal implants meet the below given estimates of With the above psychophysical estimates, a total of about minimal requirements for useful artiWcial vision. 500 electrodes would be necessary for useful artiWcial Some principal technical and biophysical problems have vision. been solved for visual prostheses in recent years (Chow & Chow, 1997; Eckmiller, 1997; Hesse, Schanze, Wilms, & 1.3. Intensity and time resolution Eger, 2000; Humayun, Probst, Juan, McCormick, & Hickingbotham, 1994; Humayun et al., 1996, 1999, 2003; Retinal ganglion cells transmit retinal visual information Normann, Maynard, Guillory, & Warren, 1996; Normann, to higher visual centers—in normal vision and with retinal  R. Eckhorn et al. / Vision Research 46 (2006) 2675–2690 2677 implants (independent of epi- or sub-retinal stimulation). existing broad knowledge about the visual systems of cats Single ganglion cells code, to a Wrst approximation, tempo- (e.g., Gilbert, 1993; Hubel & Wiesel, 1962; Tusa, Palmer, & ral sequences of local contrasts (intensities) within their Rosenquist, 1978; overview: Orban, 1984), we used this spe- receptive Welds by spike rate modulations, based on the syn- cies to estimate the achievable resolution in human visual aptic preprocessing in the network of inner and outer reti- perception with sub- and epi-retinal implants. Most impor- nal neurons. This means that the coding resolutions of tantly, for identical visual stimuli, human visual perception intensity/contrast and time are intimately related. As the has been shown to correlate with the receptive Weld proper- maximal spike rate of a ganglion cell is limited, the number ties of visual cortical neurons in lightly anaesthetized mon- of diVerent successive intensity values that can be transmit- keys and cats. Correlations were demonstrated for a variety ted in the absence of noise in terms of Shannon information of perceptual and neural thresholds of visual cortical neu- (Eckhorn & Pöpel, 1975, 1981; Eckhorn, Grüsser, Kröller, rons, including those for luminance and color contrast, Pellnitz, & Pöpel, 1976; Eger et al., 2005; Rieke, Warland, Xicker frequency, movement direction, and velocity (e.g., Gil- de Ruyter van Steveninck, & Bialek, 1998; Shannon, 1948) bert & Wiesel, 1990; Grind van de, Grüsser, & Lunkenhei- is mainly determined by the amount of time available for mer, 1973; Heydt von der & Peterhans, 1989; Knierim & van transmitting a single contrast value. For example, if a neu- Essen, 1992; Salzman, Britten, & Newsome, 1990; Wachtler, ron can discharge maximally 32 action potentials during Sejnowski, & Albright, 2003; overview: Tovee, 1996). From 333 ms (average duration of ocular Wxation) it can theoreti- these observations and retinal lesion experiments in primary cally code one out of 32 diVerent intensity values within visual cortex, one can conclude that perception of visual each of the 3 Wxation intervals per second (or 5 bit of infor- details requires the activation of neurons in this area (e.g., mation in 333 ms which equals 15 bit/s). This resolution Darian-Smith & Gilbert, 1995). From the time course of their seems suYcient for most visual Wxation tasks in which spa- activations by a focal retinal stimulus impulse estimates of tial details and high resolution of contrasts play a promi- perceptual temporal resolution should be possible (Dinse & nent role. If, on the other hand, a pedestrian wants to cross Krüger, 1994; Grüsser & Creutzfeld, 1957; Rager & Singer, a road in heavy traYc, local contrast resolution is of minor 1998; review in: Bullier, Hupe, James, & Girard, 2001). In importance, and may be reduced to only 4 diVerent values addition, the spatial extent of the cortical activations to such (resembling 2 bit). However, in such a situation temporal stimuli can give conservative estimates of spatial visual reso- resolution is of vital importance, and needs to be markedly lution. This is possible because the lower visual cortical areas increased to enable eVective coding. Our example neuron (particularly area 17/V1) are retinotopically well organized can at best signal 2 bit by 1–4 spikes per time window of so that spatial proWles of cortical activations can directly be 41.7 ms. Hence, within one second it can transmit 24 times related to retinal and visual space (Adams & Horton, 2003; 2 bit equal to 48 bit/s (again assuming the absence of noise). Angelucci, Levitt, Walton, Hupé, & Bullier, 2002; Tusa et al., This example demonstrates that a reduction in intensity 1978). These measures were veriWed in our investigation by resolution (from 5 bit to 2 bit per sample or frame) results in light stimulation of the retina before implantation of a concomitant increase of temporal resolution (from 3 to 24 electrodes combined with recordings of cortical activities samples per second) and an overall increase in the rate of from the identical positions used during electrical information from 15 bit/s to 48 bit/s. It is important to keep stimulation. in mind that the signals carrying the information are in Wrst approximation spike densities (rates) that can be read out 2. Methods by the succeeding stages of the visual system at quite diVer- ent speeds (integration window) with the consequence that Stimulation experiments with cortical electrode recordings were per- an increase in temporal resolution reduces the contrast res- formed in 9 adult cats in 13 sessions, and with optical cortical recordings olution while overall rates of information can increase. It is in 4 adult cats (3–5 kg). All experiments were done in accordance to the guidelines of the European Communities Council Directive (86/609/EEC) probable that the visual centers adaptively change the tem- and were approved by oYcial German Animal Care and Use Committees poral resolution (Agmon-Snir & Segev, 1993) according to following the NIH Principles of Laboratory Animal Care (Publication 86– the current visual situation which, in turn, aVects the avail- 23, revised 1985) and the ARVO guidelines. able contrast resolution. Retina implants have to generate spike density coding so that the visual centers can use their 2.1. Preparations Xexible strategy in which the trade-oV between intensity Preparations for cortical microelectrode and optical recordings were and time resolution is continuously optimized. similar as reported in detail elsewhere (Kisvarday, Buzas, & Eysel, 2001; Schanze et al., 2002; Wilms et al., 2003). BrieXy, adult cats received an initial 1.4. Estimates of spatial and temporal resolutions from anesthesia with ketamine hydrochloride (Ketanest, 10 mg/kg) and xylazine anesthetized cats hydrochloride (Rompun, 1 mg/kg) that was about 2 h later maintained by a mixture of N2O/O2 (70%/30%) and halothane (0.3–0.8%, optical imaging experiments) or isoXuorane (0.5–1.5%, microelectrode recording experi- Before extensive investigations of perceptual resolution ments) using oro-tracheal intubation for artiWcial respiration. Eye move- with retina implants are carried out in blind humans, we con- ments were minimized by i.v. infusion of alcuronium chloride (0.1 mg/(kg h)). sidered it ethically appropriate to test and optimize stimula- End-tidal CO2 (3–4%), blood pressure (100–140 mmHg), and body tempera- tion with retinal implants in animal models. Based on the ture (38–39 °C) were monitored continuously. 2678 R. Eckhorn et al. / Vision Research 46 (2006) 2675–2690 Semi-chronic preparations for cortical microelectrode recordings were capacity. At the other end of the foil strip a plug was soldered to the con- made with initial surgery and anesthesia (induced and prolonged) corre- tact pads to allow connection with the computer controlled stimulator sponding to the acute preparations except for the use of oro-tracheal intu- device (STG 1008, Multi Channel Systems, Reutlingen, Germany). At the bation instead of tracheal cannulation for artiWcial ventilation. For head end of each experimental session, sclera and conjunctiva were sutured, all Wxation and the attachment of a recording chamber over the craniotomy wounds topically medicated, and infections were prevented by injections at V1/V2 positions, two bolts were implanted in cavities of the forehead. In of penicillin (i.m.). The animals recovered rapidly so that after a few weeks some cases, an optically controlled laser coagulation of the peripheral ret- repetitive investigations of the same eye could be performed (Gekeler ina was performed to reduce the risk of a retinal detachment due to vitreal et al., 2004). tractions after insertion of stimulation electrodes. The refraction of the non-implanted eye was corrected for the viewing 2.4. Stimulation and recording distance of the visual stimulation screen at 1.3 m in front of the eyes. 2.4.1. Epi- and sub-retinal stimulation 2.2. Insertion of electrodes for epi-retinal stimulation For epi-retinal stimulation generally rectangular constant current impulses were applied (»200–250 s duration per polarity; §1–150 A). After lateral canthotomy at one eye the conjunctiva was incised They consisted of charge balanced pairs of a negative and positive phase (»1 mm), and a sclerostomy performed behind the limbus, suYcient for (cathodic Wrst; controlled constant current sources developed by T. insertion of the epi-retinal Wbre-microelectrode stimulation arrays. Alter- Schanze, manufactured by M. Eger, Uni-Marburg, Physics Department). natively, epi-retinal foil electrode arrays (polyimide–platinum thin-Wlm In some cases, bursts of 2–10 such impulse-pairs were applied (0.4–2.0 ms electrodes) were implanted after lensectomy and vitrectomy through cor- burst duration). In optical imaging experiments, trains of biphasic stimuli neal incisions and placed on the retinal surface adjacent to the fovea cen- were also applied (see below). For sub-retinal stimulation (see, e.g., tralis. The position of the foil electrode array was stabilized with Fig. 1B) biphasic impulses (anodic Wrst) with amplitudes up to §50 A PerXuordecalin. were applied against a ground electrode. Stimulus duration was 0.5–1.0 ms Insertion of Wber- and thin-Wlm electrodes for epi-retinal stimula- per phase applied at intervals of ISI D 305 ms. tion was made as described in detail by Schanze et al. (2002). BrieXy, the Before optical imaging the visual cortex, the function of the retinal Wber electrodes (80 m shaft diameter; Reitböck, 1983) were ground to implant was tested by electrode recordings from the optic tract, made with expose a metal (PtW) cone tip (»20 m diameter, »30 m height) a concentric bipolar tungsten electrode (SNEX-100, Rhodes Medical, allowing safe charge delivery of up to 20 nC per impulse. The Wber elec- Woodland Hills, CA, USA). Penetrations were performed at Horsley– trodes were axially positioned singly under visual control using a com- Clarke coordinates A14/L2 just behind the optic chiasm. For positioning puter driven, adapted Wber electrode manipulator (Eckhorn & Thomas, of the recording electrode at the correct depth, recordings were made dur- 1993; 1 mm concentric bundle of 3 or 7 electrode guide tubes) attached ing 1 Hz stroboscopic light Xash stimulation. The optimal position of the to another specially developed manipulator with 11 degrees of freedom electrode was reached, when the maximal Xash evoked Weld potentials (NeuroPhysics, University of Marburg). This system allowed fast, pre- were obtained and the electrode was Wxed at this position using dental cise, and Xexible electrode positioning through the small scleral opening cement. To compare the visual response amplitude to the electrically (1.1 mm). In several experiments (see Section 3), thin-Wlm arrays (plati- evoked responses and to judge the relative eYcacy of stimulation at the num islands of 100 m diameter on polyimide) were used for stimula- diVerent electrodes in the epi-retinal array, short, single biphasic electrical tion (Stieglitz, Beutel, Schuettler, & Meyer, 2000). For low energy stimuli (4–78 A; balanced current; waveform: 250 s negative; 250 s oV; activation, the electrodes were brought in direct contact with the inner 250 s positive) were delivered at 1 Hz via all single electrodes of the limiting membrane under visual guidance. To control correct position- implant against ground (electrode No. 19, see Fig. 4A). ing, multiple unit activity (MUA) of retinal ganglion cell Wbers was With sub-retinal stimulation, optic tract responses were evoked by recorded via the stimulation electrodes. electrical stimulation (§50 A, biphasic current pulses as described above). Electrodes 1, 2, 3, and 4, respectively, served as diVerent elec- 2.3. Insertion of electrodes for sub-retinal stimulation trodes, the combined electrodes 5–8 of the implant were used as ground. The small Weld potentials were averaged from 128 repetitive stimuli (see For a safe introduction of sub-retinal implants, the approach with Fig. 4B). access through the vitreous body of the eye (ab interno) as published by Cortical electrode recordings were made in area 17 and 18 (Horsley– Peyman et al. (1998) as well as Sachs et al. (1999) was used. In short, a Clarke coordinates: A2-P7, L0.5-L3) with up to 16 microelectrodes. For modiWed pars-plana-vitrectomy approach was used in the eyes after pupil technical details, see Wilms et al. (2003). BrieXy, linear arrays of Wber elec- dilation with atropine 1%. The high reXectivity of the tapetum lucidum trodes (2–3 M at 1 kHz) were used and signals were extracted in real time required no endoillumination, thereby enabling two port vitrectomy. Two from each electrode’s broad band recording: single unit spike activity sclerotomies were made 6.0 mm posterior to the limbus. After partial vit- (SUA), multiple unit spike activity (MUA), and local Weld potentials rectomy between sclerotomy and area centralis, a 31 gauge cannula (Visi- (LFP, 1 or 10–140 Hz). SUA was captured as events whereas the continu- tec, Sarasota, FL) was used to create a localized retinal bleb by injecting a ous amplitude signals MUA and LFP were sampled at 500 Hz. Alterna- small amount of balanced salt solution; the bleb was enlarged by injecting tively, the broad band raw signals (1–4000 Hz) were sampled at 20 kHz for Healon® (Pharmacia, Stockholm, Sweden). A 2.5-mm retinotomy was oV-line Wltering to SUA, MUA, and LFP, enabling a better rejection of the then made circumferentially at the temporal portion on the bleb. The foil stimulation artifacts (Schanze et al., 2002). All signals were stored on a was introduced into the vitreous cavity with an intravitreal end-gripping hard disk for oV-line data evaluations. forceps. After the foil was inserted through the retinotomy into the sub- retinal space, it was forwarded about 1 mm outside the created bleb under 2.5. Receptive Welds the retina near the area centralis (Volker et al., 2006). The stimulation array consisted of thin-Wlm platinum electrodes The receptive Welds (RF) of the recorded cortical neurons were deter- mounted on a Xexible polyimide foil (Fraunhofer Instit. Biomed. Engin., mined qualitatively using a hand held projector and quantitatively with St. Ingbert, Germany; model RS8-50; Stieglitz et al., 2000). The Xexible foil computer generated stimuli presented on a monitor (100 Hz frame rate, strip was made of 50 m thin polyimide with 8 substrate-integrated, insu- high contrast), composed of random m-sequences (multifocal RF-mea- lated golden connection lanes terminating in a 2 by 4 array at the end of surements, Sutter, 2001). After simultaneous on-line mapping of all corti- the strip with an even spacing of 330 m (see Fig. 4). Rectangular openings cal RFs, the positions of the retinal RFs were determined by the same (100 m £ 100 m) in the insulation layer at the terminals of the lanes MUA recording method. The tip positions of the stimulation electrodes deWne the size of the stimulation electrodes. The gold electrodes were cov- and retinal landmarks (blind spot and area centralis) were marked by rear ered with a thin layer of platinum to enhance the safe charge injection projection onto a tangent screen using a custom designed laser projection  R. Eckhorn et al. / Vision Research 46 (2006) 2675–2690 2679 A 5 ˚ vis. angl e B 2 ˚ vis. angl e N=101 time (1 mm retina) (0.4 mm retina) a b 100µA (ms) 0.2 x a b 3.0 mm cortex post. 1 2 3 0.5 N=202 time 5 50µA (ms) normalized amplitude 6 4.3 mm cortex 7 c d e 8 9 10 11 13 14 15 ant. 0 50 100 150 0 50 100 150 04 08 0 0 40 80 04 08 0 time (ms) time (ms) time (ms) Fig. 1. Time courses of cortical population activities (LFP) in response to single-site retinal stimulation. (A) Responses to epi-retinal impulse stimulation (N D 101 identical stimulus repetitions) recorded simultaneously at 13 diVerent locations by a linear electrode array (0.3 mm equi-distant pitch). The 10 ms gray band marks the primary cortical signals in direct response to the aVerent stimulus evoked population spike. (B) Responses to sub-retinal impulse stimulation (N D 202) at Wve retinal electrode positions (a–e) recorded simultaneously at six diVerent cortical locations by a linear electrode array (0.5 mm equi-distant pitch). The insets on top of column (A) and (B) depict the geometry of retinal stimulation sites and the time courses of stimulation current impulses with durations of 0.2 ms (A) and 0.5 ms (B). The retinotopic correspondence between retinal stimulation and cortical recording sites can be derived from the responses. The higher the response amplitudes the better the correspondence. This means for (Aa): best correspondence with electrode 3. For (B) the stimulation electrodes, used for the data in (Ba–e), are marked by gray background. Correspondence: electrodes used for (Ba) and (Bb): upper row left and right, respectively; for (Bc): lower row left and for (Be): lower row gray Weld right. Best retinotopic correspondence for data in (Ba and c–e) with electrode 4 (counted from top of each panel), for data in (Bb): el. 4 and 6. system. Retinal electrode positions were corrected until cortical and retinal system (Cambridge Research Systems, Rochester, UK). The test stimulus RFs suYciently overlapped. consisted of a narrow horizontal slit (0.5–3 deg width and 40 deg length) encompassing a high-contrast vertical grating (50% duty cycle) 0.6–1 cyc/ 2.6. Optical imaging of intrinsic signals deg spatial frequency moving forth and back at 1–2 Hz. Stimuli were pre- sented at visual Weld elevations encompassing the central 10 degrees and Optical imaging of intrinsic signals was carried out using the Imager displayed in a random sequence. Each data acquisition period (4.5 s during 2001 and the data acquisition software VDAQ-NT (both Optical Imaging, which the stimulus grating moved) was preceded by an inter-stimulus Germantown, NY, USA) as reported previously (Kisvarday et al., 2001). interval of 10 s when the animals viewed a blank screen (average lumi- BrieXy, a bilateral craniotomy was made between Horsley–Clarke coordi- nance of the stimulus grating). nates AP ¡6 and +12 and LM +0.5 and +7 to expose the central visual representation of visual areas 17 and 18. A round stainless steel chamber 2.6.2. Optical imaging using electrical stimuli (31 mm diameter) was mounted over the exposed cortical region, the dura- For obtaining intrinsic signal images of cortical activation due to mater was removed and the chamber Wlled with silicone oil (50 cSt viscos- electrical stimuli of the implanted electrodes a data acquisition paradigm ity, Aldrich, Milwaukee, WI, USA) and sealed with a round cover-glass. similar to that of the visual stimulation was used (4.5 s duration, 10 s inter- During data acquisition, the camera (two SMC Pentax lenses, 1:1.2, stimulus interval). During a single data acquisition period, 9 trains of current f D 50 mm, arranged in a “tandem” manner, RatzlaV & Grinvald, 1991) pulses were applied (250 ms train duration with 250 ms intervals). Each train was focused 650–750 m below the cortical surface and the cortex was illu- consisted of a barrage of 0.75 ms bipolar current pulses (§4–78 A, 250 s minated with 609 § 5 nm light (Wlter: Omega Optical, Brattleboro, VT). negative, 250 s baseline, 250 s positive repeated at 100 Hz). Imaging was carried out separately for visual stimuli through the non- operated control eye and for electrical activation via the implanted retinal 2.6.3. Analysis of the optical images electrodes. For data analysis, single condition maps (SCMs, c.f. BonhoeVer & Grinvald, 1996) were calculated by summing the images associated to a 2.6.1. Optical imaging using visual stimulation particular stimulus condition using the Winmix software (Optical Imag- Visuotopic mapping was performed using test stimuli presented on a ing, Germantown, NY, USA). The SCMs were Wltered with a Laplace Wlter video screen (SONY, Pencoed, UK) in 120 Hz non-interlaced mode (high-pass: 50 pixels, 1064 m) followed by a boxcar Wlter (low-pass: 28.5 cm in front of the cats’ eyes using the VSG Series 3 stimulus generator 5 pixels, 106 m). 2680 R. Eckhorn et al. / Vision Research 46 (2006) 2675–2690 2.7. Data analysis values, which also gives 5 £ 4 bit D 20 bit/s). Hence, temporal and intensity resolutions are inversely correlated, the higher the temporal the lower the 2.7.1. Cortical electrode recordings intensity resolution. The Shannon information measure has the advantage The population signals, MUA and LFP, were simultaneously available that the actual noise level is taken into account so that potential at all functioning cortex electrodes (»80%) and good SUA (at »25% elec- resolutions can be estimated quantitatively for single responses without trodes) were detected after the initial electrode positioning in the upper averaging. layers without any search for large spikes. Responses to identical stimulus repetitions were averaged with respect to stimulus onset to obtain peri- 3. Results stimulus time histograms (PSTHs) from which we determined the cortical response thresholds, delays, and strengths to retinal stimulation. These measures were based on average response power after scaling to the corre- 3.1. Spatial activation proWles in visual cortex sponding pre-stimulus values. In addition, LFP and MUA response power were plotted in relation to stimulation strength to estimate thresholds. 3.1.1. Microelectrode recordings in visual cortex The cortical responses evoked by single retinal impulses 2.7.2. Optical recordings of intrinsic cortical signals show diVerent time courses (e.g., Figs. 1A, a and b), depend- The optical recordings of intrinsic signals evoked by electrical retinal ing on the recording location relative to the retinal stimula- stimulation revealed two-dimensional activity patterns in a cortical area of 6 £ 8 mm. The grey values of the respective maps coded the activity on a tion site. These diVerences in time course are, in essence, a 0–255 gray scale with 0 (black) representing maximal, and 255 (white) sign of fast dynamic changes in the spatial activation pro- minimal activation. The peaks in the spatial pattern of activation were Wles. We measured the spatial width of these proWles to esti- used for estimation of spatial resolution. mate the potential spatial resolution obtainable with retina Spatial resolution was estimated from the spread of cortical activation implants. The narrowest distribution is generally present at in response to a focal retinal stimulus, deWned as the retino-cortical point spread function. It was either derived by sampling from linear electrode the Wrst aVerent cortical activations (Fig. 1A, gray time arrays with 0.3 or 0.5 mm spacing (7 or 16 microelectrodes) or by measur- slices). One millimeter diVerence in epi-retinal stimulus ing the spread of activation with optical imaging. For electrode recordings, location causes in this example a cortical oVset in the maxi- the spatial distribution of response amplitudes was Wtted by a Gaussian mal primary response of »3 mm (Figs. 1A, a vs. b). The function and its full width at half height was taken as the magnitude of the later (secondary) response components display a broader cortical point spread. In addition to this, the same procedure was used to determine the cortical point spread to visual retinal stimulation. In this distribution. case, a small focal Xash was shown at the same retinal location in the cor- The spatial activation proWles (retino-cortical point responding location of the non-implanted eye (results not shown). spread) to epi-retinal stimulation have almost equally often Most response proWles were measured with cortical microelectrode a single (Fig. 2A) or a double peak (Fig. 2B). It can also be recordings between 4° and 9° retinal eccentricity. The remaining optical noticed that the width of the activation increases slightly recordings were from 0° to 15° eccentricity. To compare these diVerent proWles, we took the magniWcation factor from the cat visual cortical maps with stimulation current (Figs. 2A and B), probably due to (thoroughly measured by Tusa et al., 1978 & Tusa et al., Tusa, Rosenquist, an increase in the number of recruited ganglion cells and & Palmer, 1979) at the recording site of every single cortical activation the related broader and stronger activation of the cortex. A proWle. For comparison, we took the individual magniWcation factors for coarse estimate of the full width at half height (FWHH) normalization to a reference eccentricity of 2° visual angle. We chose this obtained was 1.0 mm (Fig. 2A; stimulation: each phase reference at 2° because the retina implants in progress will probably span a total visual angle of about §4°, so that 2° eccentricity is just half way out 0.2 ms, negative Wrst, 24 A), 2.7 mm (2B; stimulation: each from the center. The response proWles measured with cortical electrodes phase 0.2 ms, negative Wrst, 25 A) and 1.7 mm (2C; stimula- were recorded at eccentricities between 4° and 9° in the lower visual Weld tion: each phase 0.5 ms, positive Wrst, 50 A). For a better where the magniWcation factor in cat does not change very much with comparison of these values, obtained at diVerent retinal eccentricity. The fewer optical recordings were made at eccentricities eccentricities, we normalized these FWHH values to an between 0° and 15°. This normalization seems appropriate when we assume that the size of a retinal patch, activated by a single electrode, is eccentricity of 2° visual angle by using the cortical magniW- independent of eccentricity because the spread of the electrical stimulation cation factors corresponding to each of our cortical record- Weld will not change with eccentricity. Therefore the corresponding activa- ing locations (Tusa et al., 1978; Tusa et al., 1979). tion patch in area 17 decreases with increasing eccentricity, and this eVect Normalization resulted in 1.4° (A), 3.8° (B), and 2.4° (C) of has been normalized by us. visual angle, respectively (see also Section 4). Estimates of resolutions for contrast and time were obtained with two methods. In the “direct approach,” the eVective duration and variance of averaged excitatory cortical MUA and LFP activations were related to 3.1.2. Optical recordings in visual cortex temporal and amplitude resolutions, respectively. The number of discrimi- Optical recordings revealed results that corresponded nable amplitudes was quantiWed by the diVerence between average post- well to the microelectrode recordings. While the fast and pre-stimulus amplitudes, and divided by the post-stimulus variance. In the “indirect approach,” the rate of transinformation T⬘ (in bit/s) was used response components could not be monitored separately (e.g., Eckhorn & Pöpel, 1975; Eger et al., 2005; Shannon, 1948). It deter- because of the relatively low temporal resolution of the mines the amount of information transmitted from a retinal stimulus elec- optical signals, the compound signals of optical record- trode to a cortical recording site. 1 bit/s here means that two diVerent ings revealed activation proWles similar to the late com- stimuli can be distinguished by an ideal observer on the basis of a single ponents of the microelectrode recordings. This is observation period of 1 s duration. If, for example, 20 bit/s are transmitted from a retinal stimulation electrode to a cortical recording site (a realistic demonstrated in Fig. 3 showing diVerent cortical activation example), a series of twenty 2-level discriminations (e.g., black/white) are proWles in response to diVerent single site retinal stimuli safely transmitted during one second (or 5 consecutively transmitted val- (visual stimulation (Fig. 3A), as well as epi- (Fig. 3B), and ues per second at a resolution of 24 D 16 diVerent levels; e.g., local contrast sub-retinal (Fig. 3C) electrical stimulation). In line with the  R. Eckhorn et al. / Vision Research 46 (2006) 2675–2690 2681 A 8 their oVsets with changes in retinal stimulus location were estimated. A shift of the stimulus by 1.5 mm on the 6 retina led to a shift of the maximum of cortical activation norm. amplitude 100µA by 3.72 mm (left and rightmost panels in Fig. 3B). The full T 49µA width at half height (FWHH) of the activity modulation 0.2 + 4 N=101 24µA in space was on average 1.78 mm. This FWHH is taken as 24 µV time 11µA I (ms) a measure to estimate the possible spatial resolution 2 11µA 0 40 80 (possible discrimination of two adjacent activity proWles) time (ms) 5µA at a given eccentricity along the antero-posterior axis in 0 visual area 18. We have calculated this resolution for the chosen retinal reference eccentricity of 2° (see Section 2: B 8 spatial resolution) where 1.78 mm in the visual cortex corresponds to about 2.5° visual angle (Tusa et al., 1979). 100µA 6 For these measurements stimulation currents were norm. amplitude 50µA §150 A (250 s duration of each phase was used for epi- retinal stimuli; s. Fig. 3B) and §50 A for the sub-retinal T 0.2 + 4 N=201 25µA 40 µV time 25µA stimuli (1 ms each phase; s. Fig. 3C). (ms) I 12.5µA Sub-retinal stimulation and optical recordings revealed 0 40 80 2 time (ms) similar results as obtained with epi-retinal stimulation. The examples in Fig. 3C show optically recorded activa- 0 tions in response to impulses applied to two diVerent reti- C 8 nal electrodes 0.33 mm apart. They are clearly diVerent in their cortical extent and position of the optical signals. This is demonstrated by the one-dimensional proWles in 6 norm. amplitude 50µA Fig. 3C. With a 0.33 mm shift of retinal stimulation in the immediate vicinity of the area centralis, the maximum of T 4 the cortical signal moves about 1.75 mm on the visual cor- I + time tical surface. From the 2-D distribution of the activity 120 µV (ms) 50µA N=201 0.5 2 proWles the spatial extent of activation was measured as 0 40 80 FWHH in the same way as for the epi-retinal stimulation. time (ms) 0 The average value of 1.79 mm FWHH corresponds to 0 1 2 3 4 5 approximately 2.5° visual angle at the reference retinal cortical position (mm) eccentricity of 2° (see Section 2) along the antero-poster- Fig. 2. Spatial cortical activation proWles (right panels) recorded with ior axis in cat visual area 18. This spatial resolution of the microelectrodes in response to para-centrally applied single retinal stimu- optical imaging signal is equivalent to that obtained with lation impulses of diVerent currents I. The normalized amplitudes of the epi-retinal stimulation. responses for deriving the spatial proWles were taken from the maxima of Table 1 gives an overview of the estimated spatial reso- the Wrst cortical response component after the stimulation artifact (indi- cated by circles in middle panels, showing example time courses of aver- lutions obtained with epi- and sub-retinal stimulation aged LFPs at the indicated stimulation currents I; delays of maxima 20– assessed with cortical microelecrode and optical record- 30 ms. T denotes the duration of the Wrst cortical response component). ings. Resolutions obtained with microelectrode record- (A) Epi-retinal stimulation with cone electrodes. (B) Epi-retinal stimula- ings were about a factor of two higher than those with tion with single electrode of foil array (manufactured by IBMT St. Ing- optical recordings. While we measured the highest resolu- bert, Germany). (C) Sub-retinal stimulation with foil electrode array (manufactured by NMI, Tübingen, Germany). Left panels: time courses of tions (0.68°) with cone-shaped three-dimensional elec- the stimulus current I; number at Wrst impulse indicates its duration in ms, trodes and epi-retinal stimulation, best results with Xat N denotes the number of identical stimulus repetitions used for averaging two-dimensional Wlm electrodes (0.9°) were obtained with the responses (examples shown in middle panels). sub-retinal stimulation. 3.2. Temporal properties of the retino-cortical pathway with retinotopic organisation of the visual cortex (Tusa et al., electrical activations 1978, 1979), the cortical activation was localized more posterior when the stimulation was located at lower posi- Temporal properties of the retino-cortical pathway were tions in the retina and moved progressively towards ante- analyzed with microelectrode recordings of single and multi- rior for stimulation sites at upper retinal locations. For a ple unit spikes (MUA) as well as local Weld potentials (LFP). semi-quantitative analysis of the cortical activity, two- dimensional activation proWles were computed on the 3.2.1. Spike activation patterns of retinal ganglion cells basis of the gray scale values of the images (Figs. 3B and Spike patterns of microelectrode recordings from the C). From such data the width of the distributions and optic tract and retinal stimulation by thin-Wlm arrays were 2682 R. Eckhorn et al. / Vision Research 46 (2006) 2675–2690 Right eye Fig. 3. Optically recorded cortical activations by visual (A), electrical epi- (B), and sub-retinal (C) stimulation with neighboring retinal electrodes. (A) Visual stimulation through a stationary horizontally oriented slit (0,5° width, 40° length) with a moving grating (see Section 2) positioned at vertical reti- nal eccentricities (elevation) corresponding (from left to right) to the electrode positions 2, 14, 23 in (B). The optical recordings demonstrate the spread and retinotopic displacement of visually evoked cortical activation proWles in cat area 18 (SCMs, see Section 2). (B and C) Optically recorded activation pro- Wles (SCMs) during stimulation with spatially displaced retinal electrodes. Graphs above each optical image show one dimensional activation proWles transformed from the intensity (gray scale) levels along the broken lines (minima represent maximal activation). Considerable cortical activation is repre- sented by the darkest black spots. They move successively from left to right with visual and electrical stimuli and are of similar size with visual and electro- cal stimuli. DiVerences are mainly due to the long slit visual and the focal electrical stimulation. The optical images show responses to stimulation of neighboring electrodes separated by 750 m (epi-retinal, (B); stimulation current: §150 A, 250 s for each phase) and 330 m (sub-retinal, (C); stimula- tion current: §50 A, 1 ms for each phase), and reveal activity shifts up to 3,7 mm along the anterior–posterior axis of the cortex (B). Importantly, the cor- tical magniWcation factor decreases about exponentially with increasing distance from the projection of the area centralis, and the shifts of the activated cortical locations correspond well to the published retinotopic maps (Tusa et al., 1978; Tusa et al., 1979). The exact position of the stimulation electrodes resulting in the activity images in (B and C) are shown in corresponding insets. Scale bars at optical images 1 mm in (A–C). L, lateral, A, anterior. analyzed. Fig. 4A shows example montages in which the rent amplitudes. Two important facts are demonstrated averaged population responses in the optic tract to single here. First, the currents for evoking similar response ampli- site stimulation are plotted together with the applied cur- tudes diVer by a factor of up to »20 (from §4 to §78 A;  R. Eckhorn et al. / Vision Research 46 (2006) 2675–2690 2683 Table 1 25 76 A Spatial resolutions obtained with epi- and sub-retinal stimulation 35 867 16 electrical 67 Cortical recording Epi-retinal stimulation Sub-retinal stimulation 1 12 60 stimulation 11 25 18 Optical 2.50° (thin-Wlm 2.50° (thin-Wlm 35 75 19 electrodes) electrodes) 4 13 42 ground Microelectrode array 0.68° (cone electrodes) 2 40 15 15 65 Microelectrode array 1.20° (thin-Wlm 0.90° (thin-Wlm 3 14 43 73 22 electrodes) electrodes) 8 Horizontal 9 23 61 1.30° (thin-Wlm 47 78 meridian 24 electrodes) 4 6 150 µV 5 21 63 The spatial resolution for microelectrode array recordings represents the 55 10 17 resolution of the primary response. The lower resolution obtained with 10 ms 70 optical recordings corresponds well to the spatial resolution of the late 20 69 responses of microelectrode recordings (see text). The high resolution of light stimulation 0.68° with epi-retinal stimulation were not obtained with currently avail- able and implantable electrode arrays but with singly in the eye inserted 200 µV microelectrodes having cone-shaped tips. OD 1 mm 100 ms epi-retinal stimulation). Second, the response delays (2– 3 ms) and their durations (1–2 ms) are very short, indicating that the electrical impulses activate ganglion cells directly B and evoke only a single spike per neuron (with epi- and sub-retinal stimulation, Fig. 4A). Fig. 4B shows responses 40 µV 5 4 to short light Xashes recorded for comparison with the same optic tract electrodes. The composite visually and 6 3 electrically evoked Weld responses were observed in a range 2 msec of 30–200 V. Compared with the responses to electrical 3 4 7 2 stimuli (delay: 2–3 ms, duration: 1 ms) the visual responses have much longer latencies (»20 ms) and duration (20– 8 1 35 ms) as is typical for visual population responses in the optic tract to which single ganglion cells generally contrib- 330 µm ute sequences of several spikes at a broad range of delays Fig. 4. Epi- and sub-retinal stimulation with thin-Wlm electrodes and pop- and with a higher degree of jitter compared to electrical ulation spike recordings from the optic tract. (A) Image of the retinal impulse activation. implant (inter-electrode distance 750 m) in the upper nasal quadrant of the retina overlapping area centralis. Schematic montage of thin-Wlm 3.3. Activation of cortical neurons array in front of retinal background superimposed by average response waveforms (small insets) evoked by short, single biphasic stimuli (§4 to §78 A; balanced current; waveform: 250 s negative; 250 s oV; 250 s 3.3.1. Temporal precision of primary cortical action positive) delivered at 1 Hz via single electrodes of the implant against potentials ground (19). Optic tract recordings were made with a concentric bipolar Retinal electrical stimuli evoked single impulses in gan- tungsten electrode (SNEX-100, Rhodes Medical, Woodland Hills, CA, glion cells with a repetition jitter of