On the Future of Transportation in an Era of Automated and Autonomous Vehicles PDF

On the future of transportation in an era of automated and autonomous vehicles P. A. Hancocka,b,1, Ill...

On the future of transportation in an era of automated and autonomous vehicles P. A. Hancocka,b,1, Illah Nourbakhshc, and Jack Stewartd a Department of Psychology, University of Central Florida, Orlando, FL 32816; bInstitute for Simulation and Training, University of Central Florida, Orlando, FL 32826; cThe Robotics Institute, Carnegie Mellon University, Pittsburgh, PA 15213; and dWired Magazine, San Francisco, CA 94107 Edited by Baruch Fischhoff, Carnegie Mellon University, Pittsburgh, PA, and approved November 1, 2018 (received for review April 3, 2018) Automated vehicles (AVs) already navigate US highways and more extensive than a simple change in the journey between the those of many other nations around the world. Current questions immediate origin and the destination. For example, in coming years it about AVs do not now revolve around whether such technologies may not be necessary for individuals to own a car, especially when should or should not be implemented; they are already with us. they can summon one from a circulating fleet using a simple portal Rather, such questions are more and more focused on how such such as a smartphone application and being fully confident that it will technologies will impact evolving transportation systems, our arrive within minutes or even seconds. This sea change in vehicle social world, and the individuals who live within it and whether usage will have many knock-on effects. Some studies have suggested such systems ought to be fully automated or remain under some that up to 30% or more of traffic circling downtown streets is actually form of direct human control. More importantly, how will mobility searching for parking (3). The search could become unnecessary when the vehicle is driving itself to pick up its next user, as some itself change as these independent operational vehicles first share projections concerning Uber usage seem to suggest. Such function- and then dominate our roadways? How will the public be kept ality could free up curb space, which is becoming increasingly more apprised of their evolving capacities, and what will be the impact important for safe pick-ups and drop-offs in already congested loca- of science and the communication of scientific advances across the tions (4). These technological changes may then foreshadow a varying forms of social media on these developments? We look repurposing of parking structures or parking spaces within buildings here to address these issues and to provide some suggestions for to accommodate new housing, offices, or retail uses. Of course, the problems that are currently emerging. parking concerns are not by any means the only dimension of change. The radical changes promised by AVs will have profound and | automated vehicles future transportation infrastructure | extended effects on the general public. Some of these are | | trust in autonomy public reactions individuated technology changes we can readily anticipate; others are much less pre- dictable. On what basis will individual members of the public judge the value of such technical innovations (5)? One prom- A utomated vehicles (AVs) are coming to our roadways. They are beginning to pose problems and issues that many of the public have not yet encountered or even witnessed. The present inent issue in such a discussion is what people understand an AV to be. It may well be that the general public views such vehicles work addresses how the capacities and impacts of growing de- as not requiring any driver input whatsoever. However, this perception fails to capture many of the major differences be- grees of ground automation can be communicated to and un- Downloaded from https://www.pnas.org by 93.41.0.6 on October 21, 2024 from IP address 93.41.0.6. tween proposed AVs and the present, semiautomated on-road derstood by the general public. To accomplish this, we have vehicles. The latter provide various forms of driver assistance to sought to integrate our individual perspectives on the issue of help the driver who remains in ultimate control. Fully autono- AVs which feature, first, a science communication viewpoint that mous vehicles are designed to drive themselves. These differing focuses specifically on how the traveling public can anticipate, forms of advancing vehicles have been categorized in a hierarchy understand, and appreciate the effects of such innovation. Our which compares driver control versus vehicle control. The hier- second narrative strand features a human-centered approach to archy is described in the Society of Automotive Engineers (SAE) the on-coming penetration of AVs, looking to understand pre- levels of control (6, 7). Although we do not specifically discuss cisely how these diverse forms of full and semiautomation will be each of these levels here, it is vital to note that many public experienced by human drivers. Finally, we conclude with an assumptions about advanced vehicle capabilities may be mis- analysis of the technical challenges guiding the possible features of placed. Thus, individuals may well assume that such AVs possess this wave of automation and prospective autonomy in future much more intelligence and operational capacity than is actually transportation. All three levels—public communication, human– the case. Such assumptions may prove critical, if not fatal. machine interaction, and technical feasibility—coact to sculpt the Some of the most evident proximal impacts will be on jobs and coming forms of transportation. The resulting system promises to associated commuting patterns. The driverless car has the po- be strikingly different from its traditional and contemporary form, tential to make its human controller as redundant as the horse which has come to be accepted as the status quo for almost a became for the horseless carriage. Truck and taxi drivers may century. Shared discourse, including public communications per- well have to find new forms of employment, some perhaps su- taining to this disruptive evolution, is critical to our collective pervising these individual vehicles from remote control call understanding of the future we may be able to create. centers. However, jobs in the new transportation sector may well Current Communications About Self‐Driving Cars As of March 2018, 52 companies possessed permits to test au- This paper results from the Arthur M. Sackler Colloquium of the National Academy of Sciences, “The Science of Science Communication III” held November 16–17, 2017, at the tonomous vehicles on the roads of the State of California alone National Academy of Sciences in Washington, DC. The complete program and audio files (1, 2). Self-driving vehicles represent a fast-paced field of mod- of most presentations are available on the NAS Web site at www.nasonline.org/ ern technology, as companies compete for dominance in this Science_Communication_III. important field of emerging transportation capacity. Neverthe- Author contributions: P.A.H., I.N., and J.S. wrote the paper. less, relatively few members of the traveling public have yet ex- The authors declare no conflict of interest. perienced trips in an autonomous vehicle. This personal inexperience This article is a PNAS Direct Submission. can make it difficult for the general populace to judge the potential utility, for good or bad, of such vehicles. The advent of the driverless Published under the PNAS license. 1 car is usually portrayed as both labor saving and accident reducing. To whom correspondence should be addressed. Email: [email protected]. However, the societal impact of these mobile robots will certainly be Published online January 14, 2019. 7684–7691 | PNAS | April 16, 2019 | vol. 116 | no. 16 www.pnas.org/cgi/doi/10.1073/pnas.1805770115  COLLOQUIUM PAPER diminish, as they have in other sectors radically changed by au- the lens of legal proceedings associated with point failures (17). tomation and now emerging machine autonomy (8). It is true However, this emphasis may create inappropriate public per- that some jobs will be created, e.g., in maintaining such fleets of ceptions regarding the safety dimension of these vehicles (13). autonomous vehicles, and access to employment for those in An informed society is important if the public is going to make economically depressed regions could be improved with AV rational choices regarding cars with no human oversight. Such transport services. Studies (9–11) show that, in general, societal choices must be informed by scientific understanding of the changes resulting from the introduction of these innovations are concerns at issue and must be clearly communicated to a con- likely to be extensive. Of course, it is likely that many human- cerned public (18). For example, exactly how safe will such driven vehicles will remain on the roadways for some decades to technologies have to be? In 2016 37,461 people died in motor come. For those who still choose to own their own vehicle, that vehicle crashes on the roads of the United States (19). Should vehicle need not be parked and taking up space for 22 hours a AVs be twice as safe, which would mean they kill only 18,000 day. It could be out earning money by giving rides to others. City people per year, or should they be 10 times as safe? Does it transit agencies need to consider the arrival of the driverless car matter who specifically is killed, or is this simply a matter of now, when plans for future transit projects are in the pipeline. absolute numbers? Who should be regarded as responsible in Does a costly subway extension still make sense in light of these conditions between AVs and human-controlled vehicles? emerging transport options? Driverless cars can provide mobility The engineers developing driverless cars must consider to those who cannot physically drive, such as children, the dis- weighty moral questions. Typically, these are considered as var- abled, or the frail elderly. However, for such populations the iations on the ethical thought experiment known as the “trolley problems of ingress into and egress from the vehicle remain, problem.” Should a driverless car swerve and injure one pedes- emphasizing that mobility is more than just the car journey alone. trian if the alternative is to continue straight ahead and injure Fuller, augmented mobility is a social amenity that can prevent the greater numbers? Should a driverless car protect the occupant loneliness, depression, and failing quality of life that attend iso- above all else or sacrifice the human on board for the greater lation and immobility. Perhaps such AVs will lure passengers off good in such circumstances? Most specifically, exactly how do we buses, deleteriously impacting the economics of bus operations in codify these respective ethical and moral principles into a soft- PSYCHOLOGICAL AND COGNITIVE SCIENCES urban areas. These represent only a limited set of the foreseeable ware assembly often created by multiple designers and code changes; more widespread and radical change is promised. developers? We believe that such problems are much more Such enormous changes, when coupled with a still relatively complex than the simple dichotomy expressed in the trolley limited public awareness resulting from the somewhat con- problem. It is virtually certain that the resolution of such issues strained distribution of the details of the precise nature of the must go well beyond the structure of the programming itself to technology’s development, means that it is important for all in- achieve full public acceptance (20). A study that posed such volved professionals to disseminate their work and clearly ar- questions to several hundred workers via Amazon’s Mechanical ticulate the limits of their own research. Of course, with such Turk service showed that the public remains conflicted about complex issues considerable uncertainty remains about the such issues (21). People are in favor of cars that sacrifice the manner in which such innovations will make both short- and passenger to save other people but would not want to be pas- long-term impacts (12). There are also further limits on science, sengers in a car programmed in that way. These choices are the and technology communication is itself changing rapidly, espe- types of scenario that engineers consider when designing their cially with the rise of the various forms of social media. Efforts to artificially intelligent machines, but a determinative public dis- provide greater opportunities for public scrutiny of science now course and communication surrounding these issues remains Downloaded from https://www.pnas.org by 93.41.0.6 on October 21, 2024 from IP address 93.41.0.6. include avenues for scientists to publish in more popular and lacking. Of course, at this moment most people still lack direct accessible outlets (13, 14). It may be that the days of the formal experience of AVs. Risk research suggests that experience and scientific journal are in relative decline. As the progressive forces information will lead to people starting to accept AVs, but this of speed and utilitarianism affect the processes of research, the propensity does not always hold in all contexts (20, 22). Such need to frame results in terms that the public can comprehend acceptance is contingent upon the swirl of public opinion, and grows accordingly. For example, Google search terms for au- such views are founded on how people experience their own tonomous vehicles spike when bad news is published, such as interactions with essentially all forms of current technology. following Uber’s fatal collision between a test vehicle and a pe- destrian in March 2018 (15). The public hears details regarding Humans and Machines accidents, but these details are not balanced by more in-depth As perhaps the primary conduit of the physical expression of communications regarding the underlying technical causes or the human freedom and certainly freedom of movement, trans- systems-level advantages of self-driving technology in particular portation in its various forms plays a critical role in virtually all cases and overall. These advantageous aspects include such di- human societies. Mobility is arguably a human right, and when mensions as energy savings, overall traffic flow efficiency, mo- access to such a facility is diminished or denied, the associated bility for the disabled, and improvements in social cohesion, to quality of life can suffer significantly (23). Especially in the larger name only a few. Indeed, a recent report has observed that the and later-developing nations of the world, the availability of general reaction to autonomous vehicle crashes is likely to be ready transport has shaped the fabric, infrastructure, and, to over-emphasized compared with the reaction to collisions in some degree, even the culture of whole countries. Now the very human-operated vehicles and that recent survey data indicate nature of such transportation is changing (24). Since the time of that confidence in AVs is actually slipping (16). the camel, the donkey, and the horse, humans have occupied the The recent fatal Uber crash came at a critical time for the seat of control (25). Humans both have decided on the strategic nascent self-driving vehicle sector. Less-than-perfect cars are mission (i.e., the desired destination) and have exercised tactical being sent onto the roads by companies that have spent billions command (i.e., the momentary control of the animal or vehicle). on research and development and are betting on their success. Of course, this has not been ubiquitously the case. We have al- Uber, Waymo, and others are conducting tests in Arizona, where ways had systems of transport in which a single driver, pilot, or regulators have taken a hands-off approach to autonomous ve- captain exercises control while many others, sometimes num- hicles; the public there and in other states is being subjected to bering in the thousands, are passengers who simply have a pas- mass testing without possessing sufficient background contextual sive role rather than any form of active control. There have been knowledge to understand the risks and benefits of such a public rudimentary forms of automation in many transport realms al- experiment. The consent of the public is largely indirect and most from the very beginnings of each technology. Such degrees implicit, since legislative deliberations on these technologies of automation have increased in sophistication across the decades. rarely access a public referendum. One of the few ways in which Many segments of the transportation system have evolved in this the general public presently encounters such vehicles is through manner with automation becoming an appreciable portion of their Hancock et al. PNAS | April 16, 2019 | vol. 116 | no. 16 | 7685  technical delivery. Now, however, we are creating and implementing forms of technology that intend to relieve humans of all momentary tactical control and even threaten to supersede all strategic control as well (26). As such, the present expansion is more a disruptive system change than a gradual and controlled evolution. Because disruptions effect broad change, a myriad of issues concerning the future of transport are embedded in many more broadly based questions about the role and impact of automated and autonomous systems in human life more generally (6, 27). Today’s vehicles are already technology-heavy semiautono- mous systems built, in turn, by other high-tech systems engi- neering and production processes. In some circumstances, the coming AVs might well appear to be robots built by robots, al- though this is not proving ubiquitously to be the case, since many manufacturers now appear to be rediscovering the advantages of human workers. As capital costs of fabrication replace labor costs in the long arc of technical productivity enhancement, we are witnessing not simply a transfer of momentary vehicle control but also a logistical tail effect in which the production and design of those vehicles also shows shifts in decision-making authority. This latter shift of power nominally pits increasingly capable computer-mediated technologies (28, 29) against humans who are largely restricted by the limitations of their inherent capac- ities to assimilate ever greater tranches of information as tech- Fig. 1. A contrast between the rates of progress in capability of humans and nological, robotic progress continues apparently unabated (30). machines over the recent industrialized epoch. Arguments that technical sys- However, the form of progress that constantly features facets of tems augment human abilities notwithstanding, the generalized curves are growing machine superiority is not without its own subtleties and revealing. Those arguing for augmentation treat the human–machine system caveats. For example, some vehicle manufacturers have found as the unit of analysis. However, it is clear that human and machine still remain that the vision of fully automated production proves less efficient fairly differentiated entities, despite efforts to combine them symbiotically. and sustainable than one in which humans and machines work The equivalent start capacities and the exact nature and timing of the crossing alongside each other. Like the operations of the innovative ve- point remain highly controversial issues, as does the future path in the com- hicles themselves, this could well imply that the human role is binatorial vector of progress shown in the shaded area of the figure (56, 78). not purely vestigial but rather is one that coevolves along with AVs may represent one technology that sees a continuing physical and func- the nature of the technological innovations involved. Such a tional separation between humans and machines. proposition argues that the development of automation, rather than being a separation of human and technology, might actually represent an on-going symbiosis between humans and the tech- growth of automation does not result in any simple one-for-one nology they create (Fig. 1). replacement of each individual human worker, even within the Downloaded from https://www.pnas.org by 93.41.0.6 on October 21, 2024 from IP address 93.41.0.6. Previous contrasts of human performance and machine per- purview of any particular domain of specific production. Rather formance have largely been set within the competitive rather the relationship is more complex, with a variegated interplay of than the cooperative perspective. Perhaps the seminal example augmentation and replacement contingent upon the innovative of such a contrasting comparison derives from the now classic technology and the fiscal and socio-technical drivers that found Fitts list (31, 32). In the Fitts report, the authors juxtaposed a such developments (40). In general, automation changes the series of capabilities in which either the machine or the human fundamental nature of the remaining human work. Automation excelled and which by implication was performed better by one may do this in time increments that cause both benefit and or the other. This list and those that have subsequently followed hardship. These effects will be felt for decades and then across it (33) have been characterized as “MABA–MABA” (Machines generations as economies adapt to the new added capacities (41). Are Better At–Men Are Better At) types of analyses (Fig. 2) Today, these dynamics and influences are reaching most evidently (34). As indicated, the essence of such comparison is one of into the multi-trillion dollar social domain of transportation. contrast, not cooperation. Some have suggested that the very way One of the proximal problems we now face concerns a quickly in which these comparisons are framed is incorrect (35). Tech- approaching and ever greater mix of multiple forms of vehicle nologies can and often do augment inherent human abilities control on mutually shared road systems (12, 42). For the fore- (36); thus rather than an explicit replacement of human knowledge seeable future many vehicles will be solely and exclusively con- and ability by more efficient machine functions, a partnership be- trolled by human drivers. Indeed, with the enthusiasts’ concern tween them is sought. Notwithstanding these arguments, two for vintage vehicles, such manually controlled machines may con- simple curves exhibit the contrasting rates of change in human tinue to be present even in highly automated contexts. However, and machine capability over time (Fig. 1). From this perspective, ever more frequently interspersed with these manually driven the relative rates of change mean that machines threaten to vehicles will be systems with some degree of shared human– supersede increasingly greater swathes of human capacity on an machine control. Indeed, we are witnessing these systems enter ever-accelerating curve. This eventuality is argued notably by into full-production vehicles at the present. In these vehicles, technology optimists who reference increasing computer and humans may temporarily drive while the vehicle assumes control robot capability and an associated decrease in the price of their on putatively more predictable multilane freeways or on major operations (37). However, it must be acknowledged that pre- road arteries, e.g., where traffic contraflow is separated by bar- cisely where we currently stand on these generalized curves has riers. The latter driving situation, which overwhelmingly de- always been a matter of debate (38). mands precise control over lateral and longitudinal positioning, Despite the apparently contrasting trends illustrated in Fig. 1, is especially well suited to current automation capacities (43). all is not dissonance between humans and technology, of which However, among this mix of manual and semiautomated control, AVs are one of the more prominent recent incarnations. For fully autonomous vehicles will also emerge. This gradual turn to instance, it is clear that only with technology can our limited semiautomated and fully automated entities brings particular planetary resources support a world population that soon issues into focus (44). Fully automated systems will be under promises to exceed eight billion persons (39). Generally, the permanent computer control, never seeking human input and 7686 | www.pnas.org/cgi/doi/10.1073/pnas.1805770115 Hancock et al.  COLLOQUIUM PAPER regulation, as indeed they do for virtually all forms of emerging technology. How to reconcile the speed of legislation and the speed of technological innovation has yet to be resolved. As a general proposition, human beings are effective in dis- tilling the attributions of others. Indeed, the cohesion of society depends upon rational attribution and common ground as- sumptions (53). Humans driving on the roads together engage in an implicitly choreographed “dance” in which, even though some prove less than completely sensitive, the general collective functions remarkably well. However, the foundational, implicit rules are not always evident. For example, people in different Ability to detect small amount of visual Ability to respond quickly to control countries and people who come from different cultures drive by or acous c energy. signals, and to apply great force smoothly and precisely. different implicit and sometimes explicit rules; one evident ex- Ability to perceive pa erns of light or ample is the side of the road on which they drive. Such cultural, sound. Ability to perform repe ve, rou ne tasks. contextual attributions can be highly problematic for strangers to Ability to improvise and use flexible procedures. Ability to store informa on briefly and that region (54). Driverless vehicles are the ultimate strangers in then to erase it completely. our midst. They currently lack the required etiquette to operate Ability to store very large amounts of informa on for long periods and to Ability to reason deduc vely, including as human beings do (55). Unlike the tourist, who at least can recall relevant facts at the appropriate computa onal ability. depend upon certain common human assumptions, AVs have me. Ability to reason handle highly complex highly limited access to the implicit rules of the road and access Ability to reason induc vely. opera ons, i.e., to do many different only the explicit rules with which they have been programed. things at once. They have even less access to social conventions at the human– Ability to exercise judgment. human interaction level. Although these latter behaviors might conceivably be programed into AVs, there seem to be few efforts PSYCHOLOGICAL AND COGNITIVE SCIENCES Fig. 2. The comparative lists generated by Fitts and his colleagues (31) in a 1951 report on the future of aviation. Although considered peripheral to the to do so in current production models. At the same time, driv- central focus of that report, the lists themselves and the associated graphic erless vehicles may be overly constrained by the formal rules of have been the subject of much discussion in the more than 60 y of their required behavior that are encoded within their logic. Thus, they existence. Most especially, much attention has been paid to whether using may be “bullied” by some aggressive human drivers. In formal direct comparisons provides the most useful strategy for the development of human–automation interaction terms, the affordance structures future human–machine collaboration. Such an approach exerts an important (56–58) of humans and autonomous vehicles are presently in- impact even today in proposals such as the SAE multiple levels of AV capacity commensurate with one another. The term “affordance,” i.e., the and associated driver interaction. Reprinted from ref. 31, with permission of relationship between an individual and the action that individual the Ohio State University Research Foundation. can take in the world, comes from the realm of ecological psy- chology. Here, we use affordance to mean what actions the driver may take given the immediate driving environment he or thus never formally possessing what we now view as an imme- she is faced with (59). For example, two-lane roadways afford diate driver. However, all these vehicular technologies will overtaking in certain conditions but not when an on-coming operate together as a community on common roadways, at least truck occupies the other lane. Downloaded from https://www.pnas.org by 93.41.0.6 on October 21, 2024 from IP address 93.41.0.6. for a number of decades. This has been referred to as the “mixed The dissonance between what the human knows of the driving equipage” or “mixed inventory” transportation state. Such mix- world and what the machine is programed to do will mean that tures of entities create their own specific problems and conflicts during the approaching transition period conflicts between hu- (45). One particularly relevant concern is the psychological man drivers and AVs are virtually inevitable (11, 60). While concept of attribution error (46). human drivers (and pedestrians) base their affordances over- Attribution is the individual’s ability to realize and recognize whelmingly on vision, this is not necessarily true for AVs, which the motivations of others (47). Human attributions usually work are informed by light detection and ranging (LIDAR), radio rather successfully. However, when the attributed motivations detection and ranging (RADAR), and vision as well as other are incorrect, they lead to cases of attribution error (48). Tra- forms of sensors. These various sensors detect other forms and ditionally, this error occurs when an individual attributes to hu- frequencies of emission and so create a perceptual “world” that man causation what is actually the result of the situation or the can be rather different from that which humans perceive. Of environment. For example, a person may have thought that course these AV sensor systems must be fused and integrated someone else’s behavior was influenced by a particular motive with each other such that the automated car “perceives” (i.e., when, in fact, environmental factors, rather than human factors assembles its sources of information in pattern-recognition as- were the proximate cause of the behavior. We have considerable semblages) the road in a very different way than humans do. This research concerning when and where people make attribution difference is probably not advisable. This divergence of these errors in relation to the motives of others and other motive forces respective human and machine affordances, and the associated (49). The open question we still face is the precise form and fre- dissonance of attribution, means that human drivers and auto- quency of attribution error when humans act in conjunction with mated cars are far from achieving the full degree of integration AVs on all segments of the roadway system (50). The roadway is a that is currently advertised on many media outlets. Of course, mutual social resource, so all actors bear shared responsibility for this issue pertains to all developing systems in which humans and common citizenship (51). In present transportation conditions, automated systems process highly divergent sources of percep- such social harmony is preserved and legally mandated by the tual input information (61). rules of the road. These rules include formal traffic-control de- This issue of attribution failure/error is almost certainly context vices and traffic laws and further involve the designed regularities specific. For example, when passing on multilane highways, there of the roadway by the civil engineers responsible for their con- is evidence that human drivers are unable to distinguish other struction. Most critically, these rules are also augmented by com- vehicles as being under either human or automated control. Ini- mon assumptions about how other drivers will behave in various tially, this might seem to indicate successful AV integration: Since driving contexts. While the design features of AVs will obviously people are unable to distinguish between the two, the AV essen- evolve during the coming years, the degree to which such features tially passes at least the surface level of the Turing test in this conform to intrinsic human attributions concerning social behav- particular context. However, in such cases indistinguishability can ior on the road is unlikely to be uniform (52). For AVs, such lead to dangerous attribution errors; e.g., humans could wrongly concerns fall in the interstices between design innovation and assume that an overtaken car has machine-level perception when, Hancock et al. PNAS | April 16, 2019 | vol. 116 | no. 16 | 7687  in fact, it is driven by a human who has blind spots (62). These However, robot sensors and actuators are not human in their dissonant attributions and their associated errors are liable to be a resilience to contextual change. Rather tritely, face-recognition problematic source of conflict as we proceed down the road to software succeeds well, except when it fails. If there are issues of autopia. Thus, such attributions will be highly contingent on the glare, backlighting, sunglasses, blur, or even busy backgrounds, general public’s perceptions of AVs, and such perceptions are then face recognition often proves unreliable. Fundamental to highly susceptible to the inconsistent coverage provided by con- autonomy is the need to preserve operational success even as temporary media outlets (63). Of course, attribution errors are not context changes and the assimilated information degrades in the only issue that beset full human–automation integration. quality and utility; however, the range of environmental changes that a self-driving car can encounter are extreme. Such real- Technical Feasibility world challenges are often set in juxtaposition to the laboratory As is always important regarding a culture-changing technical settings in today’s research parks and university laboratories innovation, historical context can help temper both cynical and where so many autonomy sensors, computational algorithms, overly optimistic reactions to our near future. Research on self- and actuators are born. Autonomous cars have indeed driven driving cars is not as new as it may seem. Autonomous driving millions of miles on California and Nevada highways. What, research was well underway in the early 1980s, and by 1996 however, does this really tell us about the suitability of driverless multiple platforms were demonstrating significant autonomous algorithms and hardware for national deployment? Road sys- capabilities, as evidenced by Carnegie Mellon University’s “No tems vary dramatically across the United States, as well as in Hands Across America” project, which demonstrated 96% au- other counties around the world. Furthermore, weather condi- tonomous driving from Pennsylvania to California along the tions in the rest of the United States are significantly worse for nation’s highways and interstates. The reason these early suc- the sensors that depend especially on visual understanding of the cesses fully two decades ago did not lead to a sea change in driving environment. Heavy rain or snow and poor road markings cause was not simply technical. As is the case today, the financial model major damage to the vision algorithms in numerous cars today. for exactly how autonomous driving would function was uncertain. Even worse we humans, as designers, engineers, and con- From sales and lease models to liability and tort, it was unclear sumers, can fall short in “engineering empathy” or “technologi- how autonomous decision making in machines could be integrated cal attribution” in understanding just how and when weather, into a legal and jurisprudence system designed entirely for human lighting, topology, and urban clutter threaten autonomy. As long action and human consequence (64). as autonomy is imperfect, the challenge to the human–systems A further, more nuanced concern has affected the optimism of integration remains. Human occupants in driverless cars under technology adopters in this area for the past two decades. The this imperative will retain a role similar to that of the ultimate autonomous car can demonstrate statistical success, traveling pilot-in-command because their intervention will eventually be thousands of miles with reasonable accident rates in the real required at some juncture as automation fails. It may be that world. Ironically, the Achilles heel of autonomous machines is human intervention need not necessarily be from inside the ve- the same as their relative advantage: the lack of tactical human hicle; instead, as in emerging drone technology, the physical decision making. When facing a highly unlikely situation that is location of the human controller can be remote from the actual nonetheless critical, the machine can face a choice point it has vehicle itself (67). Regardless of the specific spatial relations never before addressed: what if a stroller runs into the street between controller and vehicle, the human operator will require after the mother trips and falls; what if two nefarious individuals levels of effective situation awareness calibrated to ambient en- hijack the car; what if a hacker destroys the braking system; what vironmental demands at all times because the point at which any Downloaded from https://www.pnas.org by 93.41.0.6 on October 21, 2024 from IP address 93.41.0.6. if glare from a solar reflection blinds the sensors just as a truck such human intervention will be required remains unpredictable passes? Furthermore, we, as human designers and users, may as yet. Again, this raises the specter of prolonged vigilance and have absolutely no idea how that machine might respond to such its well-known decrement and response failure (68, 69). This novel scenarios. Humans have an empathetic understanding of human-as-backup architecture, which removes the person from how other humans behave when facing boundary circumstances. momentary control and instead places him or her in a supervisory We can imagine ourselves in each scenario, and we can imagine context, in many ways defeats the very idea of automation in the bounds on just how we ourselves might react, but autonomous first place. While this might be seen as a necessary transition phase cars currently express no significant forethought. At present, (6), it will be important to move to full automation quickly, so that such vehicles cannot provide hand gestures, they have no bodies the public’s expectation of hands-free and responsibility-free per- with which to jump out of the car and coax the kitten across the sonal transportation is fulfilled. Retaining the human in a state of street, and there are few prospects for such “individuation” in momentary readiness to extract the failing automation from its current and envisaged production vehicles (65). AVs are, at shortfalls is liable to be not merely unpopular but a major reason present, sufficiently foreign that the human exercise of empathy for rejecting some forms of shared control in the evolution of AVs. toward them fails us: We cannot predict how they will respond to Of course, it may be possible to engage differing forms of the unpredictable, and therein lies a social science challenge. “smart” infrastructure to share these control burdens, but this How do we consider the social ramifications of objects that will raises the difficult issue of what parts of the transport system are pervade society when we cannot even imagine how these in- publicly owned and supported versus the private vehicles that teractive, autonomous objects will respond to the boundary benefit from such social investments. Such discussions are them- conditions that will unquestionably emerge time after time? Rare selves framed within the wider context of an emerging, integrated events are by definition rare, but a one-in-a-million likelihood origin-to-destination–oriented transport system (70). Suffice it to event will happen millions of times per year if our streets are note that many sources of brittleness in the technical system re- filled with self-driving machines. This rarity by frequency prin- quire attention as momentary control passes beyond human hands, ciple is true in many domains of engineering as well as in human if indeed that is the final design goal of these advancing systems. behavior in general; if there are enough propositions, eventually Autopilot control systems in both private and commercial there must be a disposition (66); if we succeed, the unlikely will aircraft provide an informative comparison with ground AVs, but happen frequently. Engineers have not yet found and may not be there are some significant differences between the contexts. able to specify ways to characterize how our autonomous ma- First, in general, airplanes in the sky face circumstances that chines respond to the unlikely and the pathological. present somewhat fewer moment-to-moment dynamic changes A second fundamental concern stems not from any form of and a less diverse range of such challenges than those faced by direct public antipathy but from another fundamental aspect of cars on terra firma. Second, commercial aircraft still have not statistics and robotics. Robotic devices integrated into the real one but two humans ready to respond with what is anticipated to world depend critically on their sensors and actuators; these are be high levels of situation awareness. It is clearly now feasible to the devices that give them a tangible embeddedness in our world. make a fully autonomous commercial aircraft, an airliner drone. 7688 | www.pnas.org/cgi/doi/10.1073/pnas.1805770115 Hancock et al.  COLLOQUIUM PAPER However, the cost of doing so and demonstrating its absolute for the science of human–machine integration. Progress, espe- safety may well be prohibitive in our current risk-averse climate. cially in this last-mentioned area of inquiry, is not keeping pace As systems become increasingly more complex, our ability to test with that in the field of computer science or with computational their responses to all possible states diminishes accordingly, and capacity in general. The fundamental problem here is the com- we quickly reach a point at which it becomes literally impossible plexity of understanding human cognition itself. Hence, our desire to exhaustively test the system as a whole. We then are left with to innovate responsibly will demand a renewed focus on un- an alternative: to conduct a restricted set of sample-based test- derstanding the societal ramifications of innovation—innovation ing, trying to derive general conclusions from partial evaluation that is fast but whose integration into society requires us to resolve coverage and create resilience by keeping at least one or more issues that are not solely engineering challenges but are trans- humans in the loop (71). The current belief that a human or a disciplinary social concerns that can be nuanced, complex, and team of humans would be capable of innovative, on-the-job comparatively slow in developing. problem solving remains persuasive. Humans are not so obvi- ously constrained by the rules written into formal software or by Routes to Resolution the purely mechanistic limitations of the sensors and effectors of In the prior sections, we discussed some of the current and contemporary hardware. Thus it is more than probable that forthcoming challenges that face the integration of driverless during any extended transition period our semiautomated vehicles into modern transportation. Having identified such ground vehicles can and will behave very much like contempo- problems, it is incumbent upon us to offer some avenues for rary airborne systems. Whether we will need graded licenses to research and public engagement that can help us identify and operate these varying forms of automated has yet to be de- implement more robust deployments. Two vital elements here termined. Precedent argues against such a course. concern calibrated operator trust and communicated trans- Of course, this shared control and shared responsibility type of parency. For the former, design processes should seek to design activity is only one of many forms of operation that will be explicitly for appropriate levels of trust by human occupants in present at any one time. Presently, Uber states that it will field a light of the known reliability of the automation (73, 74). This team of remote human drivers, while Waymo claims to have no goal is difficult, but achieving it is critical. It is difficult because PSYCHOLOGICAL AND COGNITIVE SCIENCES need of such support services. Whether shared responsibility is we are still finding our way in understanding the contextual re- an obligatory phase of development or whether a significant liability of differing forms of automation and semiautomation portion of ground transportation can jump to full automation is offered by various manufacturers. It is critical, because if there is contingent upon many forces beyond the technical alone (e.g., insufficient human trust in autonomous and semiautonomous financial/marketing constraints). The degree to which techno- systems, there will be both little usage and chronic misuse (75). logical evolution will proceed via gradual change or jump to a The company that establishes the most effective calibration of new stable state (punctate equilibrium) (72) is still uncertain. Of trust, as a result of the highest perceived and actual reliability of course, one form of technology may exhibit gradualism while their product, will be best placed in this emerging market, that is, another allied technology vaults forward. That being so, perhaps assuming the market does not collapse if there is a catastrophic we should expect a chequered pattern of development of AVs loss of public trust. Transparency is a property of the machine by and their on-road operations. It is unlikely that the failure of one which it clearly signals both its near- and long-term intent (76). sort of AV would prejudice public opinion against all other To a degree, vehicles already possess such capacities in various forms of AV. In comparable terms, the failure of one company’s rudimentary forms. For example, displays inside the vehicle al- personal digital assistant does not spread immediately to all ready show the driver various states of engine function, gas tank Downloaded from https://www.pnas.org by 93.41.0.6 on October 21, 2024 from IP address 93.41.0.6. versions of smart phones or similar, allied technologies. status, and the like. Externally, there are existing but limited Examples of boundary conditions can often remind us of the techniques for informing other agents of vehicle intent via displays unique value of human innovativeness. Ironically, it is human such as turn signals (77). All of these express intent, albeit cur- ingenuity that can give us the examples that require a human rently on a rather low level.