Surveillance Capitalism PDF

Photo illustration by Aaron Lenchner 819461 research-article2019 NLFXXX10.1177/1095796018819461New Labor ForumZuboff New Labor Forum...

Photo illustration by Aaron Lenchner 819461 research-article2019 NLFXXX10.1177/1095796018819461New Labor ForumZuboff New Labor Forum Surveillance Capitalism and the 2019, Vol. 28(1) 10­­–29 Copyright © 2019, The Murphy Institute, City University of New York Challenge of Collective Action DOI: 10.1177/1095796018819461 https://doi.org/10.1177/1095796018819461 Shoshana Zuboff1 Keywords capitalism, surveillance, digital technologies, democracy, collective action, twenty-first-century society, social inequality, power, internet, surveillance capitalism Publisher’s note: This article has been published by permission of the author. Requests for article permissions and reprints should be directed to the author. What Is Surveillance right down to each individual member. The com- Capitalism? petition for surveillance revenues bears down on our bodies, our automobiles, our homes, and our In our time, surveillance capitalism repeats cap- cities, challenging human autonomy and demo- italism’s “original sin” of primitive accumula- cratic sovereignty in a battle for power and profit tion. It revives Karl Marx’s old image of as violent as any the world has seen. Surveillance capitalism as a vampire that feeds on labor, but capitalism cannot be imagined as something with an unexpected turn. Instead of claiming “out there” in factories and offices. Its aims and work (or land, or wealth) for the market effects are here... are us. dynamic as industrial capitalism once did, sur- Just as surveillance capitalism can no longer veillance capitalism audaciously lays claim to be conflated with an individual corporation, nei- private experience for translation into fungible ther should it be conflated with “technology.” commodities that are rapidly swept up into the Digital technologies can take many forms and exhilarating life of the market.1 Invented at have many effects, depending on the social and Google and elaborated at Facebook in the economic logics that bring them to life. The eco- online milieu of targeted advertising, surveil- nomic orientation is the puppet master; technol- lance capitalism embodies a new logic of accu- ogy is the puppet. Thus, surveillance capitalism mulation. Like an invasive species with no is not the same as algorithms or sensors, machine natural predators, its financial prowess quickly intelligence or platforms, though it depends on overwhelmed the networked sphere, grossly all of these to express its will. If technology is disfiguring the earlier dream of digital technol- bone and muscle, surveillance capitalism is the ogy as an empowering and emancipatory force. soft tissue that binds the elements and directs Surveillance capitalism can no longer be identi- them into action. Surveillance capitalism is an fied with individual companies or even with the economic creation, and it is therefore subject to behemoth information sector. This mutation democratic contest, debate, revision, constraint, quickly spread from Silicon Valley to every oversight, and may even be outlawed. economic sector, as its success birthed a bur- The primacy of economics over technology geoning surveillance-based economic order is not new, but capitalism has long found it use- that now extends across a vast and varied range ful to confound society by concealing itself of products and services.2 within the Trojan horse of technology, in order While the titanic power struggles of the twen- 1 tieth century were between industrial capital and Harvard Business School, Emerita, Boston, MA, USA labor, the twenty-first century finds surveillance Corresponding Author: capital pitted against the entirety of our societies, Shoshana Zuboff, [email protected] 12 New Labor Forum 28(1) that its excesses might be perceived as the inex- the unprecedented operations of surveillance cap- orable expression of the machines it employs. italism? How might a deeper grasp of its mecha- Surveillance capitalists are no exception. For nisms, imperatives, and production of power example, in 2009 the public first became aware illuminate both its unique threats to people and that Google maintains search histories indefi- democratic society as well as the novel chal- nitely. When questioned about these practices, lenges it presents to collective action in our age? the corporation’s former CEO Eric Schmidt explained, “... the reality is that search engines Surveillance Capitalism’s including Google do retain this information for some time.”3 In truth, search engines do not Origins and “Laws of Motion” retain, but surveillance capitalism does. Borrowed from Newton’s laws of inertia, force, Schmidt’s statement is a classic of misdirection and equal and opposite reactions, “laws of that bewilders the public by conflating commer- motion” is a metaphor that has been used to cial imperatives and technological necessity. describe the necessary and predictable features Surveillance capitalism is not inevitable but of industrial capitalism.4 Although surveillance it is unprecedented. It operates through the capitalism does not abandon established capital- instrumentation of the digital milieu, as it relies ist “laws,” such as competitive production, profit on the increasingly ubiquitous institutionaliza- maximization, productivity, and growth, these tion of digital instruments to feed on, and even earlier dynamics now operate in the context of a shape, every aspect of every human’s experi- new logic of accumulation that also introduces ence. Although it is easy to imagine the digital its own sui generis laws of motion, first discov- without surveillance capitalism, it is impossible ered and honed in the early years of Google. to imagine surveillance capitalism without the Most people credit Google’s success to its digital. In pursuing these operations, surveil- advertising model, but the discoveries that led to lance capitalism is compelled by economic Google’s rapid rise in revenue and market capi- imperatives and “laws of motion,” which pro- talization are only incidentally related to adver- duce extreme asymmetries of knowledge and tising. Google’s success derives from its ability power. Together the new capitalism and its to predict the future––specifically the future of unique production of power are as untamed by human behavior. From the start, Google had col- law as were the capitalism and economic power lected data on users’ search-related behavior as a of the Gilded Age, and its consequences, though by-product of query activity. Back then, these wholly distinct, are just as dangerous. data logs were treated as waste, not even safely or methodically stored. Eventually, the young Google’s success derives from its company came to understand that these logs could be used to teach and continuously improve ability to predict the future–– its search engine. The problem was this: Serving specifically the future of human users with effective search results “used up” all behavior. the value that users created when they inadver- tently provided behavioral data. It was a com- A century ago, Americans learned to master plete and self-contained process in which users new forms of collective action that leveraged were ends-in-themselves. All the value that users their roles as workers and customers to challenge, created was reinvested in their experience in the interrupt, and outlaw the worst injustices of raw form of improved search, a progression that I industrial capitalism. The full resources of our have called the behavioral value reinvestment democracy were eventually brought to bear in cycle. In this interaction, there was nothing “left new legislative and regulatory institutions that over,” no surplus for Google to turn into capital. subordinated the laws of supply and demand to In 2001 Google was remarkable, but it wasn’t higher order laws aimed at fostering and defend- yet capitalism––just one of many internet start- ing the conditions of a more equal, fair, and ups that boasted “eyeballs” but no revenue. humane society. Will existing forms of collective The year 2001 brought the dot.com bust and action be sufficient to tame, interrupt, or outlaw mounting investor pressures at Google. Back Zuboff 13 then advertisers selected the search term pages intelligence-based “means of production,” the for their displays. Google decided to try and more powerful are its prediction products. While boost ad revenue by applying its already sub- these processes were initially aimed at online ad stantial analytical capabilities to the challenge targeting, they are no more restricted to that of increasing an ad’s relevance to users––and application than mass production was restricted thus its value to advertisers. Operationally this to the manufacture of automobiles, where it was meant that Google would finally repurpose its first applied at scale. growing cache of “useless” behavioral data. Many of the facts I describe here are well Now the data would be used to match ads with known, but their significance has not been fully keywords, exploiting subtleties that only its appreciated or adequately theorized. Google access to behavioral data, combined with its and other surveillance platforms are sometimes analytical capabilities, could reveal. described as “two-sided” or “multisided” mar- kets, but the mechanisms of surveillance capi- Behavioral data that were talism suggest something different.5 Google once discarded or ignored were had discovered a way to translate its non-mar- ket interactions with users into surplus raw rediscovered as what I call material for the fabrication of products aimed at behavioral surplus; …this new genuine market transactions with its real cus- market exchange was not an tomers: advertisers.6 It was the translation of exchange with users but rather with private human experience situated outside the companies who understood how market into behavioral data that circulates to make money from bets on users’ within the market that finally enabled Google to convert investment into revenue and capital. future behavior. The corporation thus created out of thin air and at zero marginal cost an asset class of vital raw It’s now clear that this shift in the use of materials derived from users’ non-market behavioral data was an historic turning point. online experience. At first those raw materials Behavioral data that were once discarded or were simply “found,” a byproduct of users’ ignored were rediscovered as what I call behav- search action. Later those assets were hunted ioral surplus: data reserves that are more than aggressively, procured, and accumulated— what is required for product and service largely through unilateral operations designed improvements. Google’s dramatic success in to evade individual awareness and thus bypass “matching” ads to pages revealed the transfor- individual decision rights––operations that are mational value of this behavioral surplus as a therefore best summarized as “surveillance.” means of generating revenue and ultimately turning investment into revenue. Key to this formula was the fact that this new Google had discovered a way to market exchange was not an exchange with translate its nonmarket interactions users but rather with companies that understood with users into surplus raw material how to make money from bets on users’ future behavior. In this new context, users were no lon- for the fabrication of products ger ends-in-themselves. Instead they became a aimed at … its real customers: means to profits in new behavioral futures mar- advertisers. kets in which users are neither buyers nor sellers nor products. Instead, users are the human natu- That behavioral surplus that became the ral source of free raw material that feeds a new defining element of Google’s success was well kind of manufacturing process designed to fab- understood by its leaders. Google’s former ricate prediction products. These products are CEO Eric Schmidt credits Hal Varian’s early calculations that predict what individuals and development of the firm’s ad auctions with pro- groups will do now, soon, and later. The more viding the eureka moment that clarified the true raw materials that are fed into this new machine nature of Google’s business, “All of a sudden, 14 New Labor Forum 28(1) we realized we were in the auction business,” breadcrumbs.” The extraordinary financial referring to the automated behavioral futures power of surveillance capitalism’s hidden markets deployed in ad targeting. But Larry inventions was only revealed when Google Page is credited with a different and far more went public in 2004. At that time it became clear insightful answer to the question, “What is that on the strength of its secrets, the firm’s rev- Google?” Former Google executive Douglas enue had increased by 3,590 percent, from $86 Edwards recounts a 2001 ­ session with the million in 2001 to $3.2 billion in 2004. founders that probed their answers to that In the case of surveillance capitalism, cam- ­precise query. It was Page who ruminated, “If ouflage, euphemism, and other methodologies we did have a category, it would be of secrecy aim to prevent interruption of critical personal information....The places you’ve supply chain operations that begin with the ren- seen. Communications....Sensors are really dition of human experience and end with the cheap....Storage is cheap. Cameras are cheap. delivery of behavioral data to machine intelli- People will generate enormous amounts of gence-based production systems. These opera- data....Everything you’ve ever heard or seen or tions of secrecy-by-design turn us into exiles experienced will become searchable. Your from our own behavior, denied access to or whole life will be searchable.”7 control over knowledge derived from our expe- Page’s vision perfectly reflects the history of rience. Knowledge and power rest with surveil- capitalism as a process of taking things that live lance capital for which we are merely “human outside the market sphere and declaring their natural” resources. We are the native peoples new life as market commodities. In historian now whose tacit claims to self-determination Karl Polanyi’s 1944 grand narrative of the have vanished from the maps of our own lives. “great transformation” to a self-regulating mar- ket economy, he described the origins of this …[W]hen Google went public in translation process in three astonishing and cru- 2004… it became clear that on the cial mental inventions that he called “commod- ity fictions.” The first was that human life could strength of its secrets, the firm’s be subordinated to market dynamics and reborn revenue had increased by 3,590 as “labor” to be bought and sold. The second percent, from $86 million in 2001 to was that nature could be translated into the mar- $3.2 billion in 2004. ket and reborn as “land” or “real estate.” The third was that exchange could be reborn as To be sure, there are always sound business “money.”8 Page grasped that human experience reasons for hiding the location of your gold would be Google’s virgin wood––that it could mine. In Google’s case, an explicit “hiding strat- be extracted at no extra cost online and at a low egy” accrued to its competitive advantage, but marginal cost out in the real world, where “sen- there were other, more pressing reasons for con- sors are really cheap,” thus producing a surplus cealment and obfuscation. Douglas Edwards as the basis of a wholly new class of market writes about the corporation’s culture of secrecy: exchange. Surveillance capitalism originates in According to his account, Larry Page and this act of digital dispossession, operationalized Sergey Brin were “hawks,” insisting on aggres- in the rendition of human experience as behav- sive data capture and retention. “Larry opposed ioral data. This is the lever that moved Google’s any path that would reveal our technological world and shifted it toward profit, changing the secrets or stir the privacy pot and endanger our trajectory of information capitalism as it claimed ability to gather data.” Page questioned the pru- undefended human experience for a market dence of the electronic scroll in the reception dynamic that would encounter no impediment lobby that displays a continuous stream of in the lawless spaces of the internet. search queries, and he “tried to kill” the annual The significance of behavioral surplus was Google Zeitgeist conference that summarizes quickly camouflaged, both at Google and even- the year’s trends in search terms.9 tually throughout the internet industry, with What might the response have been back labels like “digital exhaust” and “digital then if the public were told that Google’s magic Zuboff 15 derived from its exclusive capabilities in unilat- Disguised as engines of “personalization,” digi- eral surveillance of online behavior and meth- tal assistants operate as complex supply chains ods specifically designed to override awareness for continuous automatic extraction of behav- and thus individual decision rights? Secrecy ioral surplus from human experience, its predic- was required in order to protect operations tive value ultimately realized in markets for designed to be undetectable because they took future behavior. Consider Amazon’s Alexa, things from users without asking and employed intended to become the operating system for those illegitimately claimed resources to work your life. The corporation aggressively opened in the service of others’ purposes. Alexa to third-party developers in order to That Google was able to choose secrecy is expand the “assistant’s” range of “skills,” such itself testament to the success of its own claims as reading a recipe or ordering a pizza.11 It also and an illustration of the difference between opened the Alexa platform to smart-home “decision rights” and “privacy.” Decision rights device makers from manufacturers of lighting confer the power to choose whether to keep systems to dishwashers, turning Alexa into the something secret or to share it. One can choose voice-interface for controlling home systems the degree of privacy or transparency for each and appliances. In 2015 Amazon announced situation. U.S. Supreme Court Justice William O. that Alexa would be sold as a service, known as Douglas articulated this view of privacy in 1967: “Amazon Lex,” enabling any company to inte- “Privacy involves the choice of the individual to grate Alexa’s brain into its products.12 As disclose or to reveal what he believes, what he Alexa’s senior vice president explained, “Our thinks, what he possesses.…”10 Surveillance cap- goal is to try to create a kind of open, neutral italism laid claim to these decision rights. ecosystem for Alexa … and make it as pervasive as we possibly can.”13 “As pervasive as possi- Secrecy was required in order to ble” explains why Amazon wants its Echo/ Alexa device to also function as a home phone, protect operations designed to be able to make and receive calls; why it inked an undetectable because [Google] agreement to install Echo in the nearly 5,000 took things from users without rooms of the Wynn resort in Las Vegas; and why asking… it is selling Alexa to call centers to automate the process of responding to live questions from The typical complaint is that privacy is customers by phone and text.14 By 2018 the cor- eroded, but that is misleading. In the larger poration had inked deals with home builders, societal pattern, privacy is not eroded but redis- installing its Dot speakers directly into ceilings tributed, as decision rights over privacy are throughout the house as well as Echo devices claimed for surveillance capital. Instead of and Alexa-powered door locks, light switches, many people having the right to decide how and security systems, door bells, thermostats. what they will disclose, these rights are concen- trated within the domain of surveillance capi- … Google’s Home and Amazon’s talism. Google discovered this necessary Alexa,… [d]isguised as engines element of the new logic of accumulation: it of “personalization,” … operate must declare its rights to take the information on which its success depends. These opera- as complex supply chains for tional necessities paved the way for what would continuous automatic extraction eventually become the unprecedented asymme- of behavioral surplus from human tries of knowledge over which surveillance experience… capitalists now preside. Fast forward two decades and these laws of Alexa’s skills, shape-shifting, and ubiquity motion are visible in every direction. So-called produce more and more varied interfaces with digital assistants like Google’s Home and human experience, which is then alienated from Amazon’s Alexa are frontier examples. its source, translated into behavioral data, and 16 New Labor Forum 28(1) claimed as behavioral surplus. In the process, began in the online environment and later Amazon acquires comprehensive data on peo- spread to the “real” world, constitute an extrac- ple’s actual living habits, which it learns how to tion architecture that has evolved in the direc- fabricate into behavioral predictions for sale in tion of ubiquity, just as Larry Page anticipated behavioral futures markets for real-world ser- in 2001. Under the lash of the extraction imper- vices, such as house cleaning, plumbing, or res- ative, digital instrumentation has been trans- taurant delivery.15 Amazon thus reproduces in formed into a global, sensate, computational, the real world the same logic that Google per- connected architecture of behavioral surplus fected in the virtual world, where it learned to capture and analysis, fulfilling computer scien- mine behavioral surplus from online search for tist Mark Weiser’s 1999 vision of “ubiquitous predictions of click-through rates sold into computing” memorialized in two legendary behavioral futures markets for online ad target- sentences: “The most profound technologies ing. Already forward-looking Amazon patents are those that disappear. They weave them- include the development of a “voice-sniffer selves into the fabric of everyday life until they algorithm” integrated into any device and able to are indistinguishable from it.”18 respond to hot words, such as “bought,” “dis- However, the volume of surplus became a like,” or “love” with product and service offers.16 necessary but not sufficient condition for suc- The lure of behavioral futures markets explains cess. Even the most sophisticated process of why the company joined Apple and Google in converting behavioral surplus into products that the contest for the automobile dashboard, forg- accurately forecast the future is only as good as ing alliances with Ford and BMW. The idea is to the raw material available for processing. In the host behavioral futures markets in the front seat, race for higher degrees of certainty, it became “shopping from the steering wheel” as Alexa clear that the best predictions would have to delivers restaurant recommendations or advice approximate observation. The next threshold on where to get your tires checked.17 was defined by the quality, not just the quantity, The summary of these developments is that of behavioral surplus. These pressures led to a behavioral surplus can be considered as surveil- search for new supplies of surplus that would lance assets. These assets are critical raw mate- more reliably foretell the future. This marks a rials in the pursuit of surveillance customers for critical turning point in the trial-and-error elab- the sake of surveillance revenues and their oration of surveillance capitalism and crystal- translation into surveillance capital. The entire lizes a second economic imperative—the logic of accumulation is most accurately under- prediction imperative—as the expression of stood as surveillance capitalism, which is the these competitive forces. foundational framework for a surveillance- based economic order: a surveillance economy. “The most profound technologies are those that … weave themselves Economic Imperatives into the fabric of everyday life until The accumulation of behavioral surplus is the they are indistinguishable from it.” master motion of surveillance capitalism from which key economic imperatives can be The first challenge of the prediction impera- induced. The quality of prediction products tive is economies of scope. Behavioral surplus depends on volume inputs to machine pro- must be vast, and scale remains critical, but sur- cesses. Volume surplus is thus a competitive plus must also be varied. These variations have requirement. This dynamic establishes the developed along two dimensions. The first is extraction imperative, which expresses the the extension of extraction operations from the necessity of economies of scale in surplus accu- virtual world into the “real” world of embodied mulation and depends on automated systems human experience. Surveillance capitalists that relentlessly track, hunt, and induce more understood that their future wealth would behavioral surplus. These systems, which depend on new supply routes that extend to real Zuboff 17 life on the roads, among the trees, throughout Of course, advertisers and their clients have the cities. Extension wants your bloodstream always tried to shape customer behavior and your bed, your breakfast conversation, your through priming, suggestion, and social com- commute, your run, your refrigerator, your park- parison.19 What distinguishes today’s efforts is ing space, your living room, your pancreas. that not only do they extend beyond advertis- Economies of scope also proceed along a ing, but they employ a ubiquitous digital archi- second depth dimension. The idea here is that tecture––Page’s “cheap sensors”––that is more predictive, and therefore more lucrative, finally able to automate the continuous compre- behavioral surplus can be plumbed from inti- hensive monitoring and shaping of human mate patterns of the self. These supply opera- behavior with unprecedented accuracy, inti- tions rely on emergent rendition techniques macy, and effectiveness. Economies of scale trained on new forms of surplus from facial rec- and scope are well-known industrial logics, but ognition and affective computing to voice, gait, automated economies of action are distinct to posture, and text analysis that lay bare your per- surveillance capitalism and its digital milieu. sonality, moods, emotions, lies, and vulnerabil- ities. As the prediction imperative drives deeper … [S]urveillance capitalists into the self, the value of these intimate sources of surplus becomes irresistible, and the com- gradually came to understand that petitive pressures to corner lucrative supplies the surest way to predict behavior escalate. It is no longer a matter of surveillance is to intervene at its source and capital wringing surplus from what you search, shape it. buy, and browse. Surveillance capital wants more than your body’s coordinates in time and In order to achieve these economies of action, space. Now it violates the inner sanctum, as machine processes are configured to intervene machines and their algorithms decide the mean- in the state of play in the real world among real ing of your sighs, blinks, and utterances; the people and things. These interventions are pattern of your breathing and the movements of designed to augment prediction products in your eyes; the clench of your jaw muscles; the order that they approximate certainty by “tun- hitch in your voice; and the exclamation points ing,” “herding,” and conditioning the behavior in a Facebook post once offered in innocence of individuals, groups, and populations. These and hope. economies of action apply techniques that are as varied as inserting a specific phrase into your Surveillance capitalists… want… Facebook news feed, timing the appearance of a your bloodstream and your bed, BUY button on your phone with the rise of your your breakfast conversation, endorphins at the end of a run, shutting down your car engine when an insurance payment is your commute, your run, your late, or employing population-scale behavioral refrigerator, your parking space, micro-targeting drawn from Facebook profiles. your living room, your pancreas. Indeed, the notorious manipulations of the data firm Cambridge Analytica, which scandalized Just as scale became necessary but insuffi- the world in 2018, simply appropriated the cient for higher quality predictions, the demands means and methods that are now both standard of the prediction imperative eventually encoun- and necessary operations in the surveillance tered the limitations of economies of scope. capitalism arsenal. While behavioral surplus must be vast and var- As the prediction imperative gathers force, it ied, surveillance capitalists gradually came to gradually becomes clear that economies of understand that the surest way to predict behav- scale and scope were the first phases of a more ior is to intervene at its source and shape it. The ambitious project. Economies of action mean processes invented to achieve this goal are what that ubiquitous machine architectures must be I call economies of action. able to know as well as to do. What began as an 18 New Labor Forum 28(1) extraction architecture now doubles as an exe- early theoretical physics, especially the philo- cution architecture through which hidden eco- sophical work of Max Planck. Following nomic objectives are imposed on the vast and Planck, radical behaviorism insists on the reduc- varied field of behavior. As surveillance capi- tion of human experience to observable measur- talism’s imperatives and the material infrastruc- able behavior purged of inwardness, thus tures that perform extraction and execution establishing psychological science as the objec- operations begin to function as a coherent tive study of behaving objects comparable to the whole, they produce a twenty-first-century research paradigms of the natural sciences.21 means of behavioral modification to which the Just as industrial capitalism was driven to means of production is subordinated as merely the continuous intensification of the means of one part of this larger cycle. production, so surveillance capitalists are now locked in a cycle of continuous intensification … [E]conomies of action apply of the means of behavioral modification. techniques that are as varied as Although it is possible to imagine something like a ubiquitous connected sensate computa- inserting a specific phrase into your tional architecture without surveillance capital- Facebook news feed, timing the ism, the means of behavioral modification appearance of a BUY button on depend entirely on this pervasive networked your phone with the rise of your architecture. endorphins at the end of a run… The means of behavioral The means of behavioral modification does modification …aims to produce not aim to compel conformity to or compliance behavior that reliably, definitively, with social norms, as has been the case with earlier applications of the behaviorist paradigm. and certainly leads to predicted Rather, this new complex aims to produce commercial results for surveillance behavior that reliably, definitively, and cer- customers. tainly leads to predicted commercial results for surveillance customers. The research director Economies of scale and scope ignored pri- of Gartner, the respected business advisory and vacy norms and laws, relying on weak legitima- research firm, makes the point unambiguously tion processes characteristic of meaningless when he observes that mastery of the “internet mechanisms of notice and consent (privacy of things” will serve as “a key enabler in the policies, end-user agreements, etc.) to accumu- transformation of business models from ‘guar- late decision rights in the surveillance capitalist anteed levels of performance’ to ‘guaranteed domain. Economies of action go further.22 These outcomes.’”20 This is an extraordinary state- new systems and procedures take direct aim at ment, because there can be no such guarantees individual autonomy, systematically replacing in the absence of the power to make it so. The self-determined action with a range of hidden wider complex of “the means of behavioral operations designed to shape behavior at the modification” is the expression of this gather- source. Economies of action are constructed ing power. The prospect of businesses compet- through systematic experimentation that began ing on the promise of guaranteed outcomes with apparent banalities like the A/B testing of enabled by a global digital architecture alerts us webpage design elements and eventually pro- to the force of the prediction imperative, which gressed to more complex undertakings. One now demands that surveillance capitalists make example is the secret manipulation of emotions the future for the sake of predicting it. demonstrated in Facebook’s vast experiments in The conflation of economic imperatives and shaping social behavior, about which the corpo- behavior modification at scale locates the sur- ration’s researchers concluded, “Emotional veillance capitalist project squarely in the para- states can be transferred to others via emotional digm of radical behaviorism associated with contagion, leading people to experience the B.F. Skinner, which draws on formulations in same emotions without their awareness.... Zuboff 19 Online messages influence our experience of We are learning how to write the music, and emotions, which may affect a variety of offline then we let the music make them dance.”25 behaviors.”23 Another example is the popula- What these examples share is the explicit tion-scale social herding experiments popular- aim to produce planned behavioral outcomes ized by the Google-incubated augmented reality with methods of behavioral modification that application of Niantic Labs’ Pokémon Go, in operate through unprecedented and proprietary which innocent players are herded to eat, drink, digital architectures, while carefully circum- and purchase in the restaurants, bars, fast-food venting the awareness of human targets. It is no joints, and shops that pay to play in the longer enough to automate information flows company’s behavioral futures markets.24 about us; the goal now is to automate us. This phase of surveillance capitalism’s evolution It is no longer enough to automate finally strips away the illusion that the net- information flows about us; the worked form has some kind of indigenous moral content––that being “connected” is goal now is to automate us. somehow intrinsically pro-social, innately inclusive, or naturally tending toward the Ultimately behavioral modification capabil- democratization of knowledge. Instead, digital ities are institutionalized in “innovative” com- connection is now a brazen means to others’ mercial practices in which individuals are commercial ends. Such a self-authorizing called on to fund their own domination. One power has no grounding in democratic legiti- finds digital tuning, herding, and conditioning macy, usurping decision rights, and eroding the embedded in such varied practices as the insur- processes of individual autonomy that are ance industry’s embrace of “behavioral under- essential to the function of a democratic soci- writing,” the gamification of retailing, the ety. The coda here is simple: Once I was mine. remote-control operations of automotive Now I am theirs. telematics, and the “personalized services” of the so-called digital assistants. The means of behavioral modification are The Rise of Instrumentarian the subject of creative elaboration, experimen- tation, and application, but always outside the Power awareness of its human targets. For example, There can be no behavioral modification with- the chief data scientist for a national drugstore out the power to make it so. But what is this chain described how his company designs auto- power? Just as twentieth-century scholars of mated digital reinforcers to subtly tune custom- totalitarianism once looked to nineteenth-cen- ers’ behaviors: “You can make people do things tury imperialism to explain the violence of their with this technology. Even if it’s just five per- time, it is we who now reach for the familiar cent of people, you’ve made five percent of vernaculars of twentieth-century power like people do an action they otherwise wouldn’t lifesaving driftwood. Invariably we look to have done, so to some extent there is an element Orwell’s Big Brother and more generally the of the user’s loss of self-control.” A software specter of totalitarianism as the lens through engineer specializing in the “internet of things” which to interpret today’s threats. The result is explained his company’s approach to condi- that Google, Facebook, and the larger field of tioning: “The goal of everything we do is to commercial surveillance are frequently criti- change people’s actual behavior at scale... we cized as “digital totalitarianism.”26 can capture their behaviors and identify good I admire those who have stood against the and bad. Then we develop ‘treatments’ or ‘data incursions of commercial surveillance, but I pellets’ that select good behaviors.” Another also suggest that the equation of its new power recounted the operational mechanisms of herd- with totalitarianism and the Orwellian trope ing, “We can engineer the context around a par- impedes our understanding as well as our abil- ticular behavior and force change that way.... ity to resist, neutralize, and ultimately vanquish 20 New Labor Forum 28(1) its potency. Instead, we need to grasp the spe- potential to make totalitarianism up to now cific inner logic of a conspicuously twenty- seem like sheerest anarchy.”27 In fact, all those first-century conjuring of power to which the computers are not the means to a digital hyper- past offers no adequate compass. Its aims are in totalitarianism. They are, as I think Weiser many ways just as ambitious as those of totali- sensed, the foundation of an unprecedented tarianism, but they are also utterly and pro- power that can reshape society in unprece- foundly distinct. The work of naming a strange dented ways. If instrumentarian power can form of power unprecedented in the human make totalitarianism look like anarchy, then experience must begin anew for the sake of what might it have in store for us? effective resistance and the creative power to insist on a future of our own making. There is no brother here of any kind, big or little, evil or good[,]…[i]nstead Such …self-authorizing power this new global apparatus is better has no grounding in democratic understood as a Big Other … legitimacy, … eroding the processes of individual autonomy that are While all power yearns toward totality, essential to the function of a instrumentarian power’s specific purposes and democratic society. methods are not only distinct from totalitarian- ism, they are in many ways its precise opposite. As to the new species of power, I have sug- Surveillance capitalists have no interest in murder gested that it is best understood as instrumen- or the reformation of our souls. Instrumentarian tarianism, defined as the instrumentation and power, therefore, has no principle to instruct. instrumentalization of human behavior for the There is no training or transformation for spiri- purposes of modification, prediction, monetiza- tual salvation, no ideology against which to tion, and control. In this formulation, “instru- judge our actions. It does not demand posses- mentation” refers to the ubiquitous, sensate, sion of each person from the inside out. It has computational, actuating global architecture no interest in exterminating or disfiguring our that renders, monitors, computes, and modifies, bodies and minds in the name of pure devotion. replacing the engineering of souls with the Totalitarianism was a political project that con- engineering of behavior. There is no brother verged with economics to overwhelm society. here of any kind, big or little, evil or good—no Instrumentarianism is a market project that con- family ties, however grim. Instead this new verges with the digital to achieve its own unique global apparatus is better understood as a Big brand of social domination. Totalitarianism Other that encodes the “otherized” viewpoint operated through the means of violence, but of radical behaviorism as a pervasive presence. instrumentarian power operates through the “Instrumentalization” denotes the social rela- means of behavioral modification, and this is tions that orient the puppet masters to human where our focus must shift. What passes for experience, as surveillance capital overrides social relations and economic exchange now long-standing reciprocities of market democ- occurs across the medium of this robotized veil racy, wielding its machines to transform us into of abstraction. the raw material for its own production. Instrumentarianism’s specific “viewpoint of Although he did not name it, Mark Weiser, observation” was forged in the controversial the visionary of ubiquitous computing, foresaw intellectual domain of “radical behaviorism.” the immensity of instrumentarian power as a Thanks to Big Other’s capabilities, instrumen- totalizing societal project. He did so in a way tarian power reduces human experience to mea- that suggests both its utter lack of precedent and surable observable behavior, while remaining the danger of confounding it with what has steadfastly indifferent to the meaning of that gone before: “hundreds of computers in every experience. It is profoundly, infinitely, and, fol- room, all capable of sensing people near them lowing its behaviorist origins, radically indiffer- and linked by high-speed networks have the ent to our meanings and motives. This Zuboff 21 epistemology of radical indifference produces capitalist firms and their populations. This observation without witness. Instead of an inti- absence produces the second condition, in mate violent political religion, Big Other’s way which dependency replaces reciprocity as the of knowing us yields the remote but inescapable fulcrum of this commercial project. presence of impenetrably complex systems and A first answer to the question “How do they the interests that author them, carrying individu- get away with it?” concerns a novel structural als on a fast-moving current to the fulfilment of feature of this market form that diverges sharply others’ ends. Big Other has no interest in soiling from the history of market democracy. For all itself with our excretions, but it may aggres- the failings, injustice, and violence of earlier sively hunt data on the behavior of our blood forms of modern capitalism, the necessity of and shit. It has no appetite for our grief, pain, or organic reciprocities with its populations has terror, although it welcomes the behavioral sur- been a mark of endurance and adaptability. plus that leaches from our anguish. Symbolized in the twentieth century by Ford’s Trained on measurable action, Big Other five-dollar day, these reciprocities reach back to cares only about observing what we do and Adam Smith’s original insights into the produc- ensuring that we do it in ways that are accessi- tive social relations of capitalism, in which ble to its ever-evolving operations of rendition, firms rely on people as employees and custom- reinforcement, calculation, and monetization. ers. Smith argued that price increases had to be Instrumentarianism’s radical indifference is balanced with wage increases “so that the operationalized in Big Other’s dehumanized laborer may still be able to purchase that quan- methods of evaluation that produce equivalence tity of those necessary articles which the state without equality by reducing individuals to the of the demand for labor … requires that he lowest common denominator of sameness—an should have.”30 By the 1980s, globalization and organism among organisms. neoliberal ideology, operationalized in the In the execution of economies of action, Big shareholder-value movement, went a long way Other simulates the behaviorists’ “vortex of stim- toward destroying these centuries-old reciproc- uli,” transforming “natural selection” into the ities between capitalism and its communities. “unnatural selection” of variation and reinforce- Surveillance capitalism completes the job. ment authored by market players and the competi- tion for surveillance revenues. The gentle seductive Instrumentarianism is a market voice crafted on the yonder side of this veil— Google, is that you?—herds us along the path that project that converges with the coughs up the maximum of behavioral surplus and digital to achieve its own unique the closest approximation to certainty. brand of social domination. The Challenge to Collective First, surveillance capitalists no longer rely on people as consumers. Instead, the axis of Action supply and demand orients the surveillance How do they get away with it? Dozens of sur- capitalist firm to businesses intent on anticipat- veys conducted since 2008 attest to substantial ing the behavior of populations, groups, and majorities in the United States, the European individuals. The result is that populations are Union, and around the world that reject the conceptualized as undifferentiated “users,” premises and practices of surveillance capital- who are merely the sources of raw material for ism, yet it persists, succeeds, grows, and domi- a digital-age production process aimed at a new nates, remaining largely uncontested by either business customer. Where individual consum- existing or new forms of collective action.28 In ers continue to exist in surveillance capitalist other work I have detailed sixteen conditions operations—purchasing smart appliances, digi- that enabled this new logic of accumulation to tal assistants, dolls that spy, or behavior-based root and flourish.29 Here I want to underscore insurance policies, just to name a few exam- two of these conditions: The first is the absence ples—social relations are no longer founded on of organic reciprocities between surveillance mutual exchange. In these and many other 22 New Labor Forum 28(1) instances, products and services are merely contracts.36 Should the customer refuse to agree hosts for surveillance capitalism’s data extrac- to Nest’s stipulations, the Terms of Service tion operations. For example, the concept of the indicate that the functionality and security of “smart home” has become emblematic of this the thermostat will be deeply compromised, no new asymmetry. By 2018 the global smart longer supported by the necessary updates home market was valued at $36 billion USD meant to ensure its reliability and safety. The and expected to reach $151 billion by 2023.31 consequences can range from frozen pipes to The numbers betray an earthquake beneath failed smoke alarms to an easily hacked internal their surface. Consider just one smart home home system.37 device: the Nest thermostat owned by Alphabet, The absence of consumer reciprocities is the Google holding company, and merged with complemented by the absence of employment Google in 2018.32 The Nest thermostat collects reciprocities. By historical standards the large data about its usage and environment. It uses surveillance capitalists employ relatively few motion sensors and computation to “learn” the people compared to their unprecedented compu- behaviors of a home’s inhabitants. Nest’s apps tational resources. This pattern, in which a can also gather data from other connected prod- small, highly educated workforce leverages the ucts such as cars, ovens, fitness trackers, beds.33 power of a massive capital-intensive knowl- Such systems can, for example, trigger lights if edge-production infrastructure, is called “hyper- an anomalous motion is detected, signaling scale.”38 The historical discontinuity of the video and audio recording, and even sending hyperscale business operation becomes appar- notifications to homeowners or others. As a ent by comparing seven decades of General result of the merger with Google, the thermo- Motors (GM) employment levels and market stat, like other Nest products, will be built with capitalization to recent post-IPO (initial public Google’s artificial intelligence capabilities, offering) data from Google and Facebook. (I including its personal digital “Assistant.”34 The have confined the comparison here to Google thermostat and its brethren devices create and Facebook because both were pure surveil- immense new stores of knowledge and there- lance capitalist firms even before their public fore new power—but for whom? offerings.) Wi-Fi-enabled and -networked, the thermo- stat’s intricate personalized data stores are Nest takes little responsibility for uploaded to Google’s servers. Each thermostat the security of the information it comes with a “Privacy Policy,” a “Terms of Service Agreement,” and an “End-User collects and none for how the other Licensing Agreement.” These reveal oppres- companies in its ecosystem will put sive privacy and security consequences in those data to use. which sensitive household and personal infor- mation are shared with other smart devices, From the time they went public to 2016, unnamed personnel, and third parties for the Google and Facebook steadily climbed to the purposes of predictive analyses and sales to heights of market capitalization, with Google other unspecified parties. Nest takes little reaching $532 billion by the end of 2016 and responsibility for the security of the informa- Facebook at $332 billion, without Google ever tion it collects and none for how the other com- employing more than 75,000 people or panies in its ecosystem will put those data to Facebook more than 18,000. General Motors use.35 A detailed analysis of Nest’s policies by took four decades to reach its highest market two University of London scholars concluded capitalization of $225.15 billion in 1965, when that were one to enter into the Nest ecosystem it employed 735,000 women and men.39 Most of connected devices and apps, each with its startling is that GM employed more people dur- own equally burdensome and audacious terms, ing the height of the Great Depression than the purchase of a single home thermostat entails either Google or Facebook employs at their the need to review nearly a thousand so-called heights of market capitalization. Zuboff 23 The GM pattern is the iconic story of the when Americans are informed of the ways that United States in the twentieth century, before companies gather data for targeted online ads, globalization, neoliberalism, the shareholder- 73 to 86 percent rejected such advertising.43 value movement, and plutocracy unraveled the Another substantial survey in 2015 found 91 public corporation and the institutions of what percent of respondents disagreeing that the col- historian Karl Polanyi called “the double move- lection of personal information “without my ment,” a network of “measures and policies … knowing” is a fair tradeoff for a price discount. integrated into powerful institutions designed to Fifty-five percent disagreed that it was a fair check the action of the market relative to labor, exchange for improved services.44 In 2016 land, and money.”40 Polanyi’s studies led him to PEW Research reported only 9 percent of conclude that the operations of a self-regulating respondents as very confident in trusting social market are profoundly destructive when allowed media sites with their data and 14 percent very to run free of such countervailing laws and poli- confident about trusting companies with per- cies. It was the institutions of the double move- sonal data. More than 60 percent wanted to do ment that tamed GM’s employment policies more to protect their privacy and believed there with fair labor practices, unionization, and col- should be more regulation to protect privacy.45 lective bargaining, emblematic of stable reci- Hyperscale firms have become emblematic procities during the pre-globalization decades of of modern digital capitalism, and as capitalist the twentieth century. The societal result was inventions they present significant social and predictable. In the 1950s, for example, 80 per- economic challenges, including their impact on cent of adults said that “big business” was a employment and wages, industry concentra- good thing for the country, 66 percent believed tion, and monopoly.46 In 2017 there were 24 that business required little or no change, and 60 hyperscale firms operating 320 data centers percent agreed, “the profits of large companies with anywhere between thousands and millions help make things better for everyone who buys of servers (Google and Facebook are among the their products or services.”41 largest). One hundred more data centers are expected to be online by late 2018. Microsoft [A]…survey in 2015 found 91 invested $20 billion in 2017, and in 2018 percent of respondents disagreeing Facebook announced plans to invest $20 billion in a new hyperscale data center in Atlanta. that the collection of personal According to one industry report, hyperscale information “without my knowing” firms are also building the world’s networks, is a fair tradeoff for a price especially subsea cables, which means that “a discount. large portion of the global internet traffic is now running through private networks owned or Although some critics blamed GM’s institu- operated by hyperscalers.” In 2016 Facebook tional reciprocities for its failure to adapt to global and Google teamed up to build a new subsea competition in the late 1980s, leading eventually cable between the United States and Hong to its bankruptcy in 2009, analyses have shown Kong, described as the highest-capacity trans- that chronic managerial complacency and pacific route to date.47 The surveillance capital- doomed financial strategies bore the greatest ists who operate at hyperscale or outsource to share of responsibility for the firm’s legendary hyperscale operations dramatically diminish decline, a conclusion that is fortified by the suc- any reliance on their societies as sources of cesses of the German automobile industry in the employees, and the few for whom they do com- twenty-first century, where strong labor institu- pete are largely drawn from the most-rarified tions formally share decision-making authority.42 strata of data science. Nearly seventy years later and in the absence The absence of organic reciprocities with of democratic checks on the power of surveil- people as sources of either consumers or lance capitalists, the picture is very different. employees is a matter of exceptional impor- For example, a major 2009 survey found that tance in light of the historical relationship 24 New Labor Forum 28(1) between market capitalism and democracy. In into a radical new patriotic force. Breen explains fact, the origins of democracy in both Britain that American colonists had come to depend on and America have been traced to these very the “empire of goods” imported from England, reciprocities. Even a brief glance at these histo- and that this dependency instilled the sense of a ries can help us grasp the degree to which sur- reciprocal social contract: “For ordinary people, veillance capitalism diverges from capitalism’s the palpable experience of participating in an past, a divergence in which an extreme struc- expanding Anglo-American consumer market” tural independence from people lays the foun- intensified their sense of a “genuine partnership” dation for surveillance capitalism’s unique with England. Eventually, the British Parliament approach to knowledge that we have called famously misjudged the rights and obligation of “radical indifference.” this partnership, imposing a series of taxes that In Britain, the rise of volume production and turned imported goods such as cloth and tea into its wage-earning labor force in the nineteenth “symbols of imperial oppression.” century contributed not only to workers’ eco- Breen describes the unprecedented inven- nomic power but also to a growing sense of tiveness of a political movement originating in labor’s political power and legitimacy. This pro- the shared experience of consumption, the out- duced a new sense of interdependence between rage at the violation of essential producer–con- ordinary people and elites. Economists Daron sumer interdependencies, and the determination Acemoglu and James A. Robinson show that the to make “goods speak to power.” The transla- rise of democracy in nineteenth-century Britain tion of consumer expectations into democratic was inextricably bound to industrial capital- revolution occurred in three waves, beginning ism’s dependency on the “the masses” and their in 1765, when the Stamp Act triggered popular contribution to the prosperity made possible by protests, riots, and organized resistance finally the new organization of production.48 expressed in the “nonimportation movement.” Acemoglu and Robinson conclude that the (Today we would call it a consumer boycott.) “dynamic positive feedback” between “inclu- As Breen tells it, the details of the Act were sive economic institutions” (i.e., institutions less important than the colonists’ realization defined by reciprocities) and political institu- that England did not perceive them as political tions was critical to Britain’s substantial and or economic equals bound in mutually benefi- non-violent democratic reforms. Inclusive eco- cial reciprocities. “By compromising the nomic institutions, they argue, “level the play- Americans’ ability to purchase the goods they ing field,” especially when it comes to the fight desired,” he writes, “Parliament had revealed for power, making it more difficult for elites to an intention to treat the colonists like second- “crush the masses” rather than accede to their class subjects,” levying a heavy price “on the demands. Reciprocities in economics produced pursuit of material happiness.” and sustained reciprocities in politics. In the absence of the organic reciprocities “Clamping down on popular demands,” they between producers, customers, and employees write, “and undertaking a coup against inclu- that bind populations in a shared fate, “user” sive political institutions would … destroy … dependency is the fulcrum of the surveillance [economic] gains, and the elites opposing capitalist project. Surveillance capitalism greater democratization and greater inclusive- spread across the internet just as digital com- ness might find themselves among those losing munications became the salient means of social their fortunes from this destruction.”49 participation. A 2010 BBC poll found that 79 The spread of democracy also depended on percent of people in twenty-six countries con- the reciprocities of consumption, and the sidered internet access to be a fundamental American Revolution is the outstanding example human right.50 Six years later in 2016, the of this dynamic. Historian T.H. Breen argues in United Nations Human Rights Council would his path-breaking book, The Marketplace of adopt specific language on the importance of Revolution, that it was the violation of these reci- internet access.51 In the United States, many procities that set the American Revolution into people call the emergency services number, motion, uniting disparate provincial strangers 911, on those rare occasions when Facebook is Zuboff 25 down.52 Most people find it difficult to with- anticipation” of the early-nineteenth-century draw from these utilities, and many ponder if it historian and social observer Harriet Martineau is even possible.53 The result has been an invol- who in 1833 criticized “the vulgar error of the untary merger of personal necessity and eco- aristocracy of supposing only one class of soci- nomic extraction, as the same channels that we ety to exist below that wealthy one with which rely on for daily logistics, social interaction, they are compelled by their affairs to have busi- work, education, health care, access to products ness.” This “error,” she argued, led to including and services, and much more, now double as in the single notion of “the lower classes,” supply chain operations for surveillance capi- “everybody below the wealthiest bankers— talism’s surplus flows. The result is that effec- manufacturers, tradesmen, artisans, labourers, tive social participation leads through the and paupers....”55 It would be decades until the means of behavioral modification, eroding the distinct social, economic, and political interests choice mechanisms that once adhered to the of the “laborer,” and later “the working class,” private realm––exit, voice, and loyalty. There emerged from the undifferentiated maw of the can be no exit from processes that are intention- lower classes, distinctions that both enabled ally designed to bypass individual awareness and resulted from collective action. and on which we must depend for effective daily life. Users lack reliable channels for If surveillance capitalism remains voice. Loyalty is an empty suit, as participation unchallenged…, what fresh legacy is better explained in terms of necessity, depen- of damage and regret will be dency, helplessness, resignation, the foreclo- sure of alternatives, and enforced ignorance. mourned by future generations? Now in the first decades of the twenty-first The result has been an involuntary century the distinct social, political, and eco- merger of personal necessity and nomic interests of “users” have yet to be care- economic extraction, as the same fully distinguished from the de facto conditions channels that we rely upon for of experiential dispossession, datafication, con- trol, and commodification introduced by sur- daily logistics, social interaction, veillance capitalism, reified in its behavioral work… now double as supply futures markets, and enforced by its unique and chain operations for surveillance ever-widening instrumentarian power. Unless capitalism’s surplus flows. this latency is evoked into new forms of collec- tive action, the trajectory of the digital future “User” dependency is thus a classic Faustian will be left to the new hegemon: surveillance pact in which the felt needs for effective life vie capitalism and its unprecedented asymmetries against the inclination to resist instrumentarian of knowledge and power. power’s bold incursions. This conflict produces Industrial civilization flourished at the a psychic numbing that inures users to the reali- expense of nature and threatens to cost us the ties of being tracked, parsed, mined, and modi- earth. An information civilization shaped by fied. It disposes users to rationalize the situation surveillance capitalism and its new instrumen- in resigned cynicism, shelter behind defense tarian power will thrive at the expense of human mechanisms (“I have nothing to hide”), or find nature, especially the hard-won capacities asso- other ways to stick their heads in the sand, ciated with self-determination and moral auton- choosing ignorance out of frustration and help- omy that are essential to the very possibility of lessness. In this way, surveillance capitalism a democratic society. The industrial legacy of imposes a fundamentally illegitimate choice climate chaos fills us with dismay, remorse, and that twenty-first-century individuals should not fear. If surveillance capitalism remains unchal- have to make, and its normalization leaves lenged as the dominant form of information users dancing in their chains.54 capitalism in our time, what fresh legacy of These chains mark the frontier of twenty- damage and regret will be mourned by future first-century collective action. A historical par- generations? By the time you read these words, allel is instructive. Polanyi notes the “prophetic the reach of this new form will have grown, as 26 New Labor Forum 28(1) more sectors, firms, start-ups, app developers, 11. Kevin McLaughlin et al., “Bezos Ordered and investors mobilize around this one plausi- Alexa App Push,” Information, November 16, ble version of information capitalism. This 2016, https://www.theinformation.com/bezos- mobilization and the resistance it engenders ordered-alexa-app-push; “The Real Reasons That Amazon’s Alexa May Become the Go-to will define a key battleground on which the AI for the Home,” Fast Company, April 8, 2016, next generation of collective action will be con- https://www.fastcompany.com/3058721/app- tested at the new frontier of power. economy/the-real-reasons-that-amazons-alexa- may-become-the-go-to-ai-for-the-home. Declaration of Conflicting Interests 12. “Amazon Lex—Build Conversation Bots,” The author declared no potential conflicts of interest Amazon Web Services, February 24, 2017, with respect to the research, authorship, and/or pub- https://aws.amazon.com/lex. lication of this article. 13. “Dave Limp, Exec Behind Amazon’s Alexa.” 14. Ryan Knutson and Laura Stevens, “Amazon and Funding Google Consider Turning Smart Speakers into Home Phones,” Wall Street Journal, February The author received no financial support for the 15, 2017, https://www.wsj.com/articles/ research, authorship, and/or publication of this article. amazon-google-dial-up-plans-to-turn-smart- speakers-into-home-phones-1487154781; Notes Kevin McLaughlin, “AWS Takes Aim at 1. Hannah Arendt, The Origins of Totalitarianism Call Center Industry,” Information, February (New York: Schocken, 2004), 198. Karl Polanyi, 28, 2017, https://www.theinformation.com/ The Great Transformation: The Political and aws-takes-aim-at-call-center-industry. Economic Origins of Our Time (Boston, MA: 15. [email protected] et al., “Apple Beacon Press, 2001). David Harvey, The New Loses Ground to Amazon in Smart Home Deals Imperialism (New York: Oxford University With Builders,” Information, accessed April 16, Press, 2005). 2018, https://www.theinformation.com/articles/ 2. For readers who want to explore these themes apple-loses-ground-to-amazon-in-smart-home- more deeply, they are elaborated in Shoshana deals-with-builders. Zuboff, The Age of Surveillance Capitalism: 16. Sapna Maheshwari, “Hey, Alexa, What Can The Fight for a Human Future at the New You Hear? And What Will You Do With It?,” Frontier of Power (New York: Public Affairs, New York Times, March 31, 2018, sec. Media, 2019). https://www.nytimes.com/2018/03/31/busi- 3. Jared Newman, “Google’s Schmidt Roasted for ness/media/amazon-google-privacy-digital- Privacy Comments,” PCWorld, December 11, assistants.html. 2009, http://www.pcworld.com/article/184446/ 17. “Alexa, Take the Wheel: Ford Models to Put googles_schmidt_roasted_for_privacy_com- Amazon in Driver Seat,” Bloomberg.com, ments.html. January 5, 2017, https://www.bloomberg.com/ 4. Ellen Meiksins Wood, The Origin of Capitalism: news/articles/2017-01-05/steering-wheel-shop- A Longer View (London: Verso, 2002), 76, 93, ping-arrives-as-alexa-hitches-ride-with-ford. 125. 18. Mark Weiser, “The Computer for the 21st 5. Jean-Charles Rochet and Jean Tirole, “Two- Century,” Scientific American, July 1999. Sided Markets: A Progress Report,” RAND 19. Roland Marchand, Advertising the American Journal of Economics 37, no. 3 (2006): 645– Dream: Making Way for Modernity, 1920-1940 647, http://www.jstor.org/stable/25046265. (Berkeley, CA: University of California Press, 6. Katherine J. Strandburg, “Free Fall: The Online 1985). Market’s Consumer Preference Disconnect,” 20. Christy Pettey, “Treating Information as an Working Paper, New York University Law and Asset,” Smarter with Gartner, February 17, 2016, Economics (New York University, October 1, http://www.gartner.com/smarterwithgartner/ 2013). treating-information-as-an-asset/. 7. Douglas Edwards, I’m Feeling Lucky (Boston: 21. Max Planck, “Phantom Problems in Science,” Houghton Mifflin Harcourt, 2011), 291. in Scientific Autobiography and Other Papers 8. Polanyi, The Great Transformation, 75–76. (New York: Philosophical Library, 2007), 52–79. 9. Edwards 2011, 340–345. 22. Margaret Jane Radin, Boilerplate: The Fine 10. (Douglas 1967; Farahany 2012). Print, Vanishing Rights, and the Rule of Law Zuboff 27 (Princeton, NJ: Princeton University Press, politics-and-democracy.html; Julian Assange, 2012); Peter Linzer, “Contract as Evil,” Hastings “The Banality of ‘Don’t Be Evil,’” New York Law Journal 66 (July 6, 2015): 971, https:// Times, June 1, 2013, https://www.nytimes. papers.ssrn.com/sol3/papers.cfm?abstract_ com/2013/06/02/opinion/sunday/the-banality- id=2627249; Kevin D. Werbach and Nicolas of-googles-dont-be-evil.html; Julian Assange, Cornell, “Contracts Ex Machina,” SSRN “Julian Assange on Living in a Surveillance Scholarly Paper (Rochester, NY: Social Science Society,” New York Times, December 4, 2014, Research Network, March 18, 2017), https:// https://www.nytimes.com/2014/12/04/opinion/ papers.ssrn.com/abstract=2936294; Mark julian-assange-on-living-in-a-surveillance- A. Lemley, “Terms of Use,” Minnesota Law society.html; Michael Hirsh, “We Are All Big Review 91 (July 19, 2006), https://papers.ssrn. Brother Now,” Politico, July 23, 2015, https:// com/abstract=917926; Robin Bradley Kar, www.politico.com/magazine/story/2015/07/ “The Challenge of Boilerplate,” Illinois Public big-brother-technology-trial-120477.html; Cory Law and Legal Theory Research Paper Series Doctorow, “Unchecked Surveillance Technology (University of Illinois College of Law, September Is Leading Us Towards Totalitarianism,” 3, 2013), http://juris.jotwell.com/the-challenge- International Business Times, May 5, 2017, of-boilerplate/; Nancy S. Kim, Wrap Contracts: http://www.ibtimes.com/unchecked-surveil- Foundations and Ramifications (Oxford, UK: lance-technology-leading-us-towards-totali- Oxford University Press, 2013). tarianism-opinion-2535230; Martin Schulz, 23. Adam D.I. Kramer, Jamie E. Guillory, and “Transcript of Keynote Speech at CPDP2016 Jeffrey T. Hancock, “Experimental Evidence of on Technological, Totalitarianism, Politics and Massive-Scale Emotional Contagion through Democracy,” Scribd, 2016, https://www.scribd. Social Networks,” Proceedings of the National com/document/305093114/Keynote-Speech-at- Academy of Sciences 111, no. 24 (June 17, Cpdp2016-on-Technological-Totalitarianism- 2014): 8788–8790, https://doi.org/10.1073/ Politics-and-Democracy. pnas.1320040111. 27. Weiser, “The Computer for the 21st Century,” 89. 24. For more on Pokémon Go and other examples 28. See, for example, Chris Jay Hoofnagle and of economies of action see the discussion in Jennifer King, “Research Report: What Zuboff, The Age of Surveillance Capitalism, Californians Understand About Privacy 2019, chapter 10. Offline” (SSRN Scholarly Paper, Rochester, 25. Zuboff, The Age of Surveillance Capitalism, NY: Social Science Research Network, 2019, p. 295. May 15, 2008), http://papers.ssrn.com/ 26. See, for example, Peter S. Menell, “2014: Brand abstract=1133075; Joseph Turow et al., Totalitarianism,” UC Berkeley Public Law “Americans Reject Tailored Advertising and Research Paper (Berkeley, CA: University of Three Activities That Enable It,” Annenberg California, September 4, 2013), http://papers. School for Communication, September 29, ssrn.com/abstract=2318492; “Move Over, 2009, http://papers.ssrn.com/abstract=1478214; Big Brother,” Economist, December 2, 2004, Joseph Turow, Michael Hennessy, and Nora http://www.economist.com/node/3422918; Draper, “The Tradeoff Fallacy: How Marketers Wojciech Borowicz, “Privacy in the Internet Are Misrepresenting American Consumers and of Things Era,” Next Web, October 18, 2014, Opening Them Up to Exploitation,” Annenberg http://thenextweb.com/dd/2014/10/18/privacy- School for Communication, June 2015, https:// internet-things-era-will-nsa-know-whats-fridge; www.asc.upenn.edu/news-events/publications/ Tom Sorell and Heather Draper, “Telecare, tradeoff-fallacy-how-marketers-are-misrepre- Surveillance, and the Welfare State,” American senting-american-consumers-and; Lee Rainie, Journal of Bioethics 12, no. 9 (2012): 36–44, “Americans’ Complicated Feelings About Social https://doi.org/10.1080/15265161.2012.69913 Media in an Era of Privacy Concerns,” Pew 7; Rhys Blakely, “‘We Thought Google Was the Research Center (blog), March 27, 2018, http:// Future but It’s Becoming Big Brother,’” Times, www.pewresearch.org/fact-tank/2018/03/27/ September 19, 2014, http://www.thetimes. americans-complicated-feelings-about-social- co.uk/tto/technology/internet/article4271776. media-in-an-era-of-privacy-concerns. ece; CPDP Conferences, Technological 29. Zuboff, The Age of Surveillance Capitalism: Totalitarianism, Politics and Democracy, 2016, The Fight for a Human Future at the New http://www.internet-history.info/media-library/ Frontier of Power (New York: Public Affairs, mediaitem/2389-technological-totalitarianism- 2019). 28 New Labor Forum 28(1) 30. Smith, The Wealth of Nations, 939–940. 39. These data are drawn from my own compilation 31. Reuters, Global Smart Homes Market 2018 of General Motors market capitalization and by Evolving Technology, Projections & employment data from 1926 to 2008; Google Estimations, Business Competitors, Cost from 2004 to 2016; and Facebook from 2012 Structure, Key Companies and Forecast to 2023, to 2016. All market capitalization values are February 19, 2018. https://www.reuters.com/ adjusted for inflation to 2016 dollars, as per the brandfeatures/venture-capital/article?id=28096. Consumer Price Index from Federal Reserve 32. Ron Amadeo, “Nest Is Done as a Standalone Economic Data, Economic Research Division, Alphabet Company, Merges with Google,” Ars Federal Reserve Bank of St. Louis. The sources Technica, February 7, 2018, https://arstechnica. used to compile these data include Standard & com/gadgets/2018/02/nest-is-done-as-a-stand- Poor’s Capital IQ (Google Market Capitalization alone-alphabet-company-merges-with-google/; and Headcount), Wharton Research Data Leo Kelion, “Google-Nest Merger Raises Services – CRSP (General Motors Market Privacy Issues,” BBC News, February 8, 2018, Capitalization), Standard & Poor’s Compustat sec. Technology, http://www.bbc.com/news/ (General Motors Headcount), Thomas Eikon technology-42989073. (Facebook Market Capitalization), Company 33. Kelion, “Google-Nest Merger Raises Privacy Annual Reports (General Motors Headcount), Issues.” and SEC Filings (Facebook Headcount). 34. “Nest to Join Forces with Google’s Hardware 40. Polanyi, The Great Transformation, 79. Team,” Google, February 7, 2018, https:// 41. Opinion Research Corporation, “Is Big w w w. b l o g. g o o g l e / t o p i c s / h a r d w a r e / Business Essential for the Nation’s Growth nest-join-forces-googles-hardware-team/. and Expansion?,” ORC Public Opinion Index 35. Grant Hernandez, Orlando Arias, Daniel (USA, August 1954); Opinion Research Buentello, and Yier Jin. “Smart nest thermo- Corporation, “Which of These Comes Closest to stat: A smart spy in your home,” Black Hat Your Impression of the Business Setup in This USA (2014), https://www.blackhat.com/docs/ Country?,” ORC Public Opinion Index (USA, us-14/materials/us-14-Jin-Smart-Nest-Thermo- January 1955); Opinion Research Corporation, stat-A-Smart-Spy-In-Your-Home-WP.pdf. “Now Some Questions about Large Companies. 36. Guido Noto La Diega, “Contracting for the Do You Agree or Disagree on Each of These?... ‘Internet of Things’: Looking into the Nest,” Large Companies Are Essential for the Nation’s Research Paper (London, UK: Queen Mary Growth and Expansion.,” ORC Public Opinion University of London, School of Law, 2016); Index (USA, June 1959); Louis Harris & Robin Kar and Margaret Radin, “Pseudo- Associates, “Which Two or Three Best Describe Contract & Shared Meaning Analysis,” Legal Most Business Corporation Leaders in the Studies Research Paper (University of Illinois Country?,” Harris Survey (Connecticut, April College of Law, November 16, 2017), https:// 1966); Louis Harris & Associates, “Compared papers.ssrn.com/abstract=3083129. with What We Have Produced in the Past in 37. Hernandez, et. al., “Smart nest thermostat: A This Country, Do You Feel That Our Present smart spy in your home,” Black Hat USA (2014). Leadership in the Field of Business Is Better, 38. James Manyika and Michael Chui, “Digital Worse or about the Same as We Have Produced Era Brings Hyperscale Challenges,” Financial in the Past?,” Harris Survey (Connecticut, June Times, August 13, 2014, http://www.ft.com/ 1968); Professor Louis Galambos, The Public intl/cms/s/0/f30051b2-1e36-11e4-bb68- Image of Big Business in America, 1880-1940: A 00144feabdc0.html?siteedition=intl#axzz3JjX Quantitative Study in Social Change (Baltimore: PNno5; “Hyperscalers Taking over the World Johns Hopkins University Press, 1975). at an Unprecedented Scale,” Data Economy 42. See, for example, Alfred D. Chandler, “The (blog), April 11, 2017, https://data-economy. Enduring Logic of Industrial Success,” Harvard com/hyperscalers-taking-world-unprecedented- Business Review, March 1, 1990, https://hbr. scale/; Paul McNamara, “What Is Hyperscale org/1990/03/the-enduring-logic-of-industrial- and Why Is It So Important to Enterprises?,” success; Susan Helper and Rebecca Henderson, n.d., http://cloudblog.ericsson.com/digital- “Management Practices, Relational Contracts, services/what-is-hyperscale-and-why-is-it-so- and the Decline of General Motors,” Journal of important-to-enterprises; Digital Realty, “What Economic Perspectives 28, no. 1 (February 2014): Is Hyperscale?,” text/html, Digital Realty, 49–72, https://doi.org/10.1257/jep.28.1.49. February 2, 2018, https://www.digitalrealty. 43. Turow et al., “Americans Reject Tailored com/blog/what-is-hyperscale/. Advertising and Three Activities That Enable It.” Zuboff 29 44. Turow, Hennessy, and Draper, “The Tradeoff 52. “911 Calls about Facebook Outage Angers L.A. Fallacy: How Marketers Are Misrepresenting County Sheriff’s Officials,” Los Angeles Times, American Consumers and Opening Them August 1, 2014, http://www.latimes.com/local/ Up to Exploitation,” Annenberg School for lanow/la-me-ln-911-calls-about-facebook-out- Communication. age-angers-la-sheriffs-officials-20140801-ht- 45. Rainie, “Americans’ Complicated Feelings about mlstory.html. Social Media in an Era of Privacy Concerns.” 53. Cecilie Schou Andreassen et al., “Development 46. David H. Autor et al., “The Fall of the Labor of a Facebook Addiction Scale,” Psychological Share and the Rise of Superstar Firms,” Reports 110, no. 2 (April 2012): 501–517, https:// SSRN Scholarly Paper (Rochester, NY: Social doi.org/10.2466/02.09.18.PR0.110.2.501-517; Science Research Network, May 22, 2017), Cecilia Cheng and Angel Yee-lam Li, “Internet https://papers.ssrn.com/abstract=2971352. Addiction Prevalence and Quality of (Real) See also, Michael Chui and James Manyika, Life: A Meta-Analysis of 31 Nations across “Competition at the Digitial Edge: ‘Hyperscale’ Seven World Regions,” Cyberpsychology, Businesses,” McKinsey Quarterly, March 2015. Behavior and Social Networking 17, no. 47. See “Hyperscalers Taking Over the World 12 (December 2014): 755–60, https://doi. at an Unprecedented Scale,” Data Economy org/10.1089/cyber.2014.0317; Adam Alter, (blog), April 11, 2