AI-anxiety.pdf

For the exclusive use of C. CHALEM, 2023. Digital Article Risk Management Generative AI-nxiety Leaders are feeling disoriented and concerned about the new technology. Here are four key risks to understand — and advice on how to address them. by Reid Blackman This document is authorized for use only...

For the exclusive use of C. CHALEM, 2023. Digital Article Risk Management Generative AI-nxiety Leaders are feeling disoriented and concerned about the new technology. Here are four key risks to understand — and advice on how to address them. by Reid Blackman This document is authorized for use only by CLAUDE CHALEM in 2023. For the exclusive use of C. CHALEM, 2023. HBR / Digital Article / Generative AI-nxiety Generative AI-nxiety Leaders are feeling disoriented and concerned about the new technology. Here are four key risks to understand — and advice on how to address them. by Reid Blackman Published on HBR.org / August 14, 2023 / Reprint H07RA1 HBR Staff; Mensent Photography/Getty Images; Midjourney Leaders across all industries are facing pressure from their boards and CEOs to figure out where a generative AI solution can be implemented. The rationale is familiar: On the one hand, there’s excitement for capitalizing on new opportunities, and on the other, a fear of falling behind the competition. But amidst the push to innovate there is also well-founded anxiety. Samsung banned use of ChatGPT after employees loaded sensitive company data onto the platform that subsequently leaked. The well-documented tendency of AI to generate discriminatory outputs applies to generative AI, too. Meanwhile, generative AI Copyright © 2023 Harvard Business School Publishing. All rights reserved. This document is authorized for use only by CLAUDE CHALEM in 2023. 1 For the exclusive use of C. CHALEM, 2023. HBR / Digital Article / Generative AI-nxiety companies are facing lawsuits: StableDiffusion, which generates images, faces a lawsuit from Getty Images, while Microsoft, GitHub, and OpenAI face down a class-action lawsuit. The people and companies responsible for this technology are also ringing alarm bells, from various “godfathers of AI” like Geoffrey Hinton and Yoshua Bengio to the likes of Sam Altman, CEO of OpenAI. Humans, they claim, could face extinction — or at least dominance by their robot overlords — in the near future. Other voices warn of the ease of creating high-quality misinformation campaigns on the eve of a U.S. presidential election while still others warn of the potential economic catastrophe caused by AI replacing human workers. In the past several months of advising enterprise clients, I’ve found corporate leaders both on the technology side and the non-technology side confused as to what they should pay attention to. The cacophony of alarms has left them disoriented and concerned, particularly given that generative AI is available to everyone within their organizations, not just data scientists. As one client remarked, spelling out why they need an AI ethical risk program: “It’s not that anything is on fire. It’s that everyone in our organization is holding a flame thrower.” What Risks Do You Really Need to Worry About? While generative AI presents many genuine ethical risks, those risks don’t apply to each and every corporation. First, even if we grant for the sake of argument that the cumulative effect of AI on the economy results in mass joblessness, it doesn’t follow that any particular business has an obligation to stop this. After all, on a standard view of the responsibilities of businesses, they do not have an obligation to hire or retain employees. It may be ethically good to keep people on when they could be replaced with more efficient and/or Copyright © 2023 Harvard Business School Publishing. All rights reserved. This document is authorized for use only by CLAUDE CHALEM in 2023. 2 For the exclusive use of C. CHALEM, 2023. HBR / Digital Article / Generative AI-nxiety less-expensive AI — and in some cases I would encourage companies to do just that — but it’s not generally considered an ethical requirement. Second, the threat of the spread of (election) misinformation is arguably one of the greatest risks modern democracies face — it certainly ranks in my top three — but most corporations are not in the business of helping individuals or businesses spread information. It’s unlikely that this risk pertains to your organization unless it’s a social media company. Third, even if we grant that AI poses an existential risk to the human race in anything like the near future, there is probably very little your organization can do about it. If you can do something about it, then by all means, please do. Now that we’ve put those to ethical risks to the side, let’s turn to the ones most companies need to face. The first thing companies need to do is ask: 1. What ethical, reputational, regulatory, and legal risks does generative AI share with non-generative AI? 2. What ethical, reputational, regulatory, and legal risks are particular to, or exacerbated by, generative AI compared to non-generative AI? As to the first question, non-generative AI can easily create biased or discriminatory outputs. It can also produce outputs in a way that cannot be explained, known as the black box problem. And non-generative AI can be trained on or create data that violates others’ privacy. Finally, many ethical risks of non-generative AI are use-case specific. That is because the kinds of ethical risks an organization faces depends upon the various contexts in which non-generative AI is deployed. Copyright © 2023 Harvard Business School Publishing. All rights reserved. This document is authorized for use only by CLAUDE CHALEM in 2023. 3 For the exclusive use of C. CHALEM, 2023. HBR / Digital Article / Generative AI-nxiety Generative AI shares all of these ethical risks with non-generative AI. Both image and text generators have demonstrated biases in their outputs. With regard to the black box problem, the data scientists themselves cannot explain how the output has gotten so good. And because these models are trained with data that was scraped from the internet, that data includes both data about individuals and data that is people’s or organization’s intellectual property. Finally, as with non-generative AI, the ethical risks of generative AI are also use-case specific. But there’s a twist. Generative AI is generalpurpose AI. This means it can be used in countless use cases across every industry. Not only that, but organizations now have thousands (if not tens or hundreds of thousands) of employees who have access to these new tools. Organizations not only need to get their hands around the use case–specific ethical risks of AI designed by data scientists and engineers, but also those countless contexts in which their employees may use an AI. This brings us to our second question: What ethical, reputational, regulatory, and legal risks are particular to, or exacerbated by, generative AI compared to non-generative AI? Generative AI’s Ethical Risks There are at least four cross-industry risks that organizations need to get a handle on: the hallucination problem, the deliberation problem, the sleazy salesperson problem, and the problem of shared responsibility. Understanding these risks in detail can help companies plan how they want to address them. The hallucination problem One significant risk related to LLMs like OpenAI’s ChatGPT, Microsoft’s Bing, and Google’s Bard is that they generate false information. Examples of how risky this is abound. Think Copyright © 2023 Harvard Business School Publishing. All rights reserved. This document is authorized for use only by CLAUDE CHALEM in 2023. 4 For the exclusive use of C. CHALEM, 2023. HBR / Digital Article / Generative AI-nxiety of doctors using an LLM to diagnose patients, consumers using an LLM to ask for financial or relationship advice, or consumers asking for information about a product, to name a few of the countless contexts in which an LLM may be deployed. There are a few significant aspects of the hallucination problem that need to be emphasized. First, checking the veracity of the claims of an LLM cannot be automated. There is no software that can run a program that checks claims against reality. Verifying the truth/falsity of a claim must be done manually. Second, people tend to trust the outputs of software programs. In fact, the tendency is so well-established there’s a name for it: “automation bias.” Thus, the manual verification that needs to be performed is something that must take place against the countervailing force of automation bias. The problem is exacerbated by the tone of authority LLMs often manifest. LLMs are not only too frequently wrong, but too frequently confidently wrong. Third, and relatedly, people are lazy and want quick answers — which is one reason to turn to an LLM in the first place — and manual checking of the veracity of the outputs can take quite a bit of effort and occurs slowly. Fourth, as the flamethrower analogy above highlights, this is a tool that literally everyone in your organization has access to. Fifth and finally, many people aren’t aware that LLMs confidently espouse false claims, making them particularly vulnerable to overreliance on the tool. Copyright © 2023 Harvard Business School Publishing. All rights reserved. This document is authorized for use only by CLAUDE CHALEM in 2023. 5 For the exclusive use of C. CHALEM, 2023. HBR / Digital Article / Generative AI-nxiety It’s important to highlight against this backdrop that merely telling employees that LLMs can output false information is not enough to stop them from automatically relying on it. Knowledge is one thing; action is another. Rationalizing that “this output is probably fine” will likely be common given automation bias, laziness, and the need for speed. Due diligence processes, compliance with those processes, and usage monitoring are needed to combat these foes, as is involving other people who may correct for someone else’s all-too-human shortcomings. The deliberation problem LLMs appear to have the power to deliberate — to present coherent reasoning that looks like thinking — but in reality, they’re generating a thin facsimile, and one that can be dangerous. Suppose a financial advisor is not sure about what to recommend and so consults an LLM for advice. The LLM might recommend a certain investment strategy, complete with the alleged reasoning behind the advice. But don’t be fooled: Even if it looks like an LLM is providing an explanation behind its output, it’s actually just generating a plausible-sounding reason, based on a process of predicting which words go together. This point is a bit subtle, so let’s take our time with it. LLMs are in the business of finding the next set of words that is maximally coherent with the words that came before it. Its outputs should make sense to the user. What’s so startling about recent LLMs like GPT-4 is that they can make sense without knowing what they’re saying. An LLM doesn’t understand its outputs. It doesn’t grasp the meanings of words. It’s a surprisingly successful (though far from perfect) next-words-predictor. This means that when you ask an LLM for an explanation for why it recommended X, it doesn’t actually give you an explanation for why Copyright © 2023 Harvard Business School Publishing. All rights reserved. This document is authorized for use only by CLAUDE CHALEM in 2023. 6 For the exclusive use of C. CHALEM, 2023. HBR / Digital Article / Generative AI-nxiety it recommended X. It predicts the next words it “thinks” coheres with the conversation that has thus far transpired. It does not articulate the reasons it recommended X because it doesn’t provide outputs for reasons. It does not deliberate or decide. It just predicts next-wordlikelihood. So it can’t give you the reasons it recommended X because its output is based on probability, not reasons. Instead, it fabricates reasons — which, to the unsuspecting user, genuinely look like the reasons behind the outputs. This generates at least two problems. First, it’s easy to get fooled by this. Suppose, for instance, a user overcomes their automation bias, laziness, and need for speed and begins to probe the LLM for a justification for its answer. Presumably the user consulted the LLM in the first place because they were unsure of the answer in a complex situation. Now the LLM patiently and authoritatively explains to the user its (alleged) rationale for its recommendation. It’s now quite easy for the user to defer to the seemingly authoritative, deliberative LLM. And now we’re back to where we were before their efforts to overcome their bias, laziness, and need for speed. Second, sometimes it matters that a person deliberates. While there are some scenarios in which performance is all that matters, there are others where it matters to us — or at least some of us — that there’s a person on the other end deliberating about how to treat us properly. In a criminal justice context, for instance, you might care not only that the judge gets the right answer but also that she engages in deliberation. Her thinking about you and your case is part of what it is Copyright © 2023 Harvard Business School Publishing. All rights reserved. This document is authorized for use only by CLAUDE CHALEM in 2023. 7 For the exclusive use of C. CHALEM, 2023. HBR / Digital Article / Generative AI-nxiety to be respected by the judge. Offloading that decision to a computer is, arguably, ethically objectionable. Similarly, we want good financial advice, but we also want to know that we’re getting that advice from someone who is actively deliberating about what is best for us. There’s a human element to relationships — particularly in high-stakes situations — that, arguably, we don’t want to get rid of. More to the point, even if you, the reader, don’t feel this way, nonetheless some of your organization’s customers likely do feel this way. In that case, as a matter of respect for their wishes to maintain that human element, not replacing deliberative people with faux-deliberative software is required. Like with the hallucination problem, the remedies for the deliberation problem are due diligence processes, monitoring, and human intervention. The sleazy salesperson problem Perhaps the best method to sell anyone anything is to talk to them. In some cases, salespeople sell while being on the up and up. In others, there’s the sleazy or slick salesperson who excels at pushing people’s emotional buttons to get them to buy things they don’t really want. In fact, during a summer job working for a car rental company, I was taught — and at the time, proudly executed on — a tactic to get people to buy car insurance (sorry — “coverage”; we weren’t allowed to say “insurance”) by triggering the renter’s fear of what could possibly go wrong in a car. Similarly, in website design, there are methods to manipulate users into, among other things, giving up on their attempt to cancel their account; these are called “dark patterns.” Now suppose some people in your organization — motivated by the familiar cocktail of financial incentives and pressure to hit certain numbers — develops an LLM sales chatbot that is very good at Copyright © 2023 Harvard Business School Publishing. All rights reserved. This document is authorized for use only by CLAUDE CHALEM in 2023. 8 For the exclusive use of C. CHALEM, 2023. HBR / Digital Article / Generative AI-nxiety manipulating people. It’s “read” all the books on how to push people’s buttons and all the books on negotiation and has been instructed to converse with consumers in a way that is commensurate with what it’s learned. This is a great recipe for undermining the trustworthiness of your organization. When consumers are systematically duped at scale by your consumer-facing LLM chatbot, you’ll lose enough trust to warrant spending more money than you earned from the trickery to gain back that trust. (Not to mention it’s just ethically gross to systemically deceive people into buying your products.) The problem of shared responsibility For the most part, generative AI models, also called “foundation models,” are built by a handful of companies. If your organization sources its generative AI from one of these companies, then your organization is likely going to “fine tune” that model. Your in-house data scientists and engineers are there to do that work. But if something goes ethically sideways when deploying your fine-tuned generative AI, the question arises: Who is responsible? The answer to this question is complicated. First, foundation models are often black boxes. This means that we — including data scientists — cannot explain how the AI arrives at its outputs given the inputs. Second, many companies that build the foundation models are not particularly transparent about the decisions that were made throughout the design, build, and validation lifecycles of the AI. For instance, they may not share what data they used to train the AI. Your organization thus faces the question: Do we have enough information from the supplier that built the foundation model such that we can do sufficient ethical, reputational, regulatory, and legal due diligence as we fine-tune and deploy the model? Copyright © 2023 Harvard Business School Publishing. All rights reserved. This document is authorized for use only by CLAUDE CHALEM in 2023. 9 For the exclusive use of C. CHALEM, 2023. HBR / Digital Article / Generative AI-nxiety Let me put it differently. Suppose your organization deploys a generative AI model and things go ethically sideways. If your organization had enough information from the foundation model supplier that they could have done tests that would have caught the problem but didn’t do those tests, then (all else equal) the responsibility lies on the shoulders of your organization. On the other hand, if your organization did not have enough information such that it couldn’t do its due diligence effectively, then the responsibility is on both the supplier and your organization. It’s on the supplier because they should have provided you with the information your organization needed to do its due diligence. It’s on your organization because either your teams didn’t realize they didn’t have enough information or they did know and decided to move forward anyway. This shows us how important a feasibility analysis is when sourcing and then fine-tuning a generative AI model. Part of that analysis must include whether your organization’s team can assess what they need to do their due diligence, whether they will get that information from the supplier, and what the benchmarks are for “sufficiently safe for deployment.” Managing This Risk In light of these problems, some companies have moved to ban use of generative AI in their organizations. This is unwise. In fact, it’s a bit like telling teenagers to be abstinent and then refusing to provide safe sex education; bad things are going to happen. This is one reason why enterprise-wide education on the safe use of generative AI — including clearly articulated and easy processes by which they can raise questions to the appropriate subject matter experts and authorities within your organization — needs to be prioritized in a way it didn’t before generative AI. Copyright © 2023 Harvard Business School Publishing. All rights reserved. This document is authorized for use only by CLAUDE CHALEM in 2023. 10 For the exclusive use of C. CHALEM, 2023. HBR / Digital Article / Generative AI-nxiety That said, generative AI’s ethical risks are not so novel that they defy existing approaches to designing and implementing an AI ethical risk program. The four problems articulated above highlight the need for additional focus on these kinds of risks, but the very basic strategies for addressing them are of a piece with strategies that apply to non-generative AI including, among other things, an ethical risk due diligence process that occurs at each stage of the AI lifecycle, an AI ethical risk committee, AI ethical risk learning and development throughout the enterprise, and metrics and KPIs to measure the rollout, compliance, and impact of your AI ethical risk program. This article was originally published online on August 14, 2023. Reid Blackman is the author of Ethical Machines (Harvard Business Review Press, 2022), the host of a podcast by the same name, and the founder and CEO of Virtue, a digital ethical risk consultancy. He advises the government of Canada on federal AI regulations and corporations on how to implement digital ethical risk programs. He has been a senior adviser to the Deloitte AI Institute, served on Ernst & Young’s AI Advisory Board, and volunteers as the chief ethics officer to the nonprofit Government Blockchain Association. Previously he was a professor of philosophy at Colgate University and the University of North Carolina, Chapel Hill. Copyright © 2023 Harvard Business School Publishing. All rights reserved. This document is authorized for use only by CLAUDE CHALEM in 2023. 11