Artificial Intelligence in K-12 Education: Ethical Challenges (PDF)

AI and Ethics (2022) 2:431–440 https://doi.org/10.1007/s43681-021-00096-7 REVIEW Artificial intelligence in education: Addressing ethical challenges in K‑12 settings Selin Akgun1 · Christine Greenhow1 Received: 9 July 2021 / Accepted: 1 September 2021 / Published online: 22 September 20...

AI and Ethics (2022) 2:431–440 https://doi.org/10.1007/s43681-021-00096-7 REVIEW Artificial intelligence in education: Addressing ethical challenges in K‑12 settings Selin Akgun1 · Christine Greenhow1 Received: 9 July 2021 / Accepted: 1 September 2021 / Published online: 22 September 2021 © The Author(s), under exclusive licence to Springer Nature Switzerland AG 2021 Abstract Artificial intelligence (AI) is a field of study that combines the applications of machine learning, algorithm productions, and natural language processing. Applications of AI transform the tools of education. AI has a variety of educational applica- tions, such as personalized learning platforms to promote students’ learning, automated assessment systems to aid teachers, and facial recognition systems to generate insights about learners’ behaviors. Despite the potential benefits of AI to support students’ learning experiences and teachers’ practices, the ethical and societal drawbacks of these systems are rarely fully considered in K-12 educational contexts. The ethical challenges of AI in education must be identified and introduced to teachers and students. To address these issues, this paper (1) briefly defines AI through the concepts of machine learning and algorithms; (2) introduces applications of AI in educational settings and benefits of AI systems to support students’ learning processes; (3) describes ethical challenges and dilemmas of using AI in education; and (4) addresses the teaching and understanding of AI by providing recommended instructional resources from two providers—i.e., the Massachusetts Institute of Technology’s (MIT) Media Lab and Code.org. The article aims to help practitioners reap the benefits and navigate ethical challenges of integrating AI in K-12 classrooms, while also introducing instructional resources that teachers can use to advance K-12 students’ understanding of AI and ethics. Keywords Artificial intelligence · K-12 education · Ethics · Teacher education 1 Introduction of AI systems reveals itself in healthcare, education, commu- nications, transportation, agriculture, and more. It is almost “Success in creating AI would be the biggest event in human impossible to live in a modern society without encountering history. Unfortunately, it might also be the last, unless we applications powered by AI [10, 32]. learn how to avoid the risks.”—Stephen Hawking. Artificial intelligence (AI) can be defined briefly as the We may not think about artificial intelligence (AI) on a branch of computer science that deals with the simulation daily basis, but it is all around us, and we have been using it of intelligent behavior in computers and their capacity to for years. When we are doing a Google search, reading our mimic, and ideally improve, human behavior. AI dom- emails, getting a doctor’s appointment, asking for driving inates the fields of science, engineering, and technology, directions, or getting movie and music recommendations, we but also is present in education through machine-learning are constantly using the applications of AI and its assistance systems and algorithm productions. For instance, AI in our lives. This need for assistance and our dependence has a variety of algorithmic applications in education, such on AI systems has become even more apparent during the as personalized learning systems to promote students’ COVID-19 pandemic. The growing impact and dominance learning, automated assessment systems to support teach- ers in evaluating what students know, and facial recogni- tion systems to provide insights about learners’ behaviors * Selin Akgun. Besides these platforms, algorithm systems are prom- [email protected] inent in education through different social media outlets, Christine Greenhow such as social network sites, microblogging systems, and [email protected] mobile applications. Social media are increasingly inte- 1 grated into K-12 education and subordinate learners’ Michigan State University, East Lansing, MI, USA 13 Vol.:(0123456789) 432 AI and Ethics (2022) 2:431–440 activities to intelligent algorithm systems. Here, we Next, we briefly define the notion of artificial intelligence use the American term “K–12 education” to refer to stu- (AI) and its applications through machine-learning and algo- dents’ education in kindergarten (K) (ages 5–6) through rithm systems. As educational and educational technology 12th grade (ages 17–18) in the United States, which is scholars working in the United States, and at the risk of similar to primary and secondary education or pre-college oversimplifying, we provide only a brief definition of AI level schooling in other countries. These AI systems can below, and recognize that definitions of AI are complex, increase the capacity of K-12 educational systems and multidimensional, and contested in the literature [9, 16, support the social and cognitive development of students 38]; an in-depth discussion of these complexities, however, and teachers [55, 8]. More specifically, applications of is beyond the scope of this paper. Second, we describe in AI can support instruction in mixed-ability classrooms; more detail five applications of AI in education, outlining while personalized learning systems provide students with their potential benefits for educators and students. Third, detailed and timely feedback about their writing products, we describe the ethical challenges they raise by posing the automated assessment systems support teachers by freeing question: “how and in what ways do algorithms manipulate them from excessive workloads [26, 42]. us?” Fourth, we explain how to support students’ learning Despite the benefits of AI applications for education, they about AI and ethics through different curriculum materials pose societal and ethical drawbacks. As the famous scientist, and teaching practices in K-12 settings. Our goal here is to Stephen Hawking, pointed out that weighing these risks is provide strategies for practitioners to reap the benefits while vital for the future of humanity. Therefore, it is critical to navigating the ethical challenges. We acknowledge that in take action toward addressing them. The biggest risks of centering this work within U.S. education, we highlight cer- integrating these algorithms in K-12 contexts are: (a) per- tain ethical issues that educators in other parts of the world petuating existing systemic bias and discrimination, (b) per- may see as less prominent. For example, the European Union petuating unfairness for students from mostly disadvantaged (EU) has highlighted ethical concerns and implications of and marginalized groups, and (c) amplifying racism, sex- AI, emphasized privacy protection, surveillance, and non- ism, xenophobia, and other forms of injustice and inequity discrimination as primary areas of interest, and provided. These algorithms do not occur in a vacuum; rather, guidelines on how trustworthy AI should be [3, 15, 23]. they shape and are shaped by ever-evolving cultural, social, Finally, we reflect on future directions for educational and institutional and political forces and structures [33, 34]. As other research that could support K-12 teachers and students academics, scientists, and citizens, we have a responsibil- in reaping the benefits while mitigating the drawbacks of AI ity to educate teachers and students to recognize the ethical in education. challenges and implications of algorithm use. To create a future generation where an inclusive and diverse citizenry can participate in the development of the future of AI, we 2 Definition and applications of artificial need to develop opportunities for K-12 students and teach- intelligence ers to learn about AI via AI- and ethics-based curricula and professional development [2, 58] The pursuit of creating intelligent machines that replicate Toward this end, the existing literature provides little human behavior has accelerated with the realization of arti- guidance and contains a limited number of studies that focus ficial intelligence. With the latest advancements in computer on supporting K-12 students and teachers’ understanding science, a proliferation of definitions and explanations of of social, cultural, and ethical implications of AI. Most what counts as AI systems has emerged. For instance, AI has studies reflect university students’ engagement with ethi- been defined as “the ability of a digital computer or com- cal ideas about algorithmic bias, but few addresses how to puter-controlled robot to perform tasks commonly associated promote students’ understanding of AI and ethics in K-12 with intelligent beings”. This particular definition high- settings. Therefore, this article: (a) synthesizes ethical issues lights the mimicry of human behavior and consciousness. surrounding AI in education as identified in the educational Furthermore, AI has been defined as “the combination of literature, (b) reflects on different approaches and curriculum cognitive automation, machine learning, reasoning, hypoth- materials available for teaching students about AI and ethics esis generation and analysis, natural language processing, (i.e., featuring materials from the MIT Media Lab and Code. and intentional algorithm mutation producing insights and org), and (c) articulates future directions for research and analytics at or above human capability”. This defini- recommendations for practitioners seeking to navigate AI tion incorporates the different sub-fields of AI together and and ethics in K-12 settings. 13 AI and Ethics (2022) 2:431–440 433 underlines their function while reaching at or above human 3 Benefits of AI applications in education capability. Combining these definitions, artificial intelligence can be Personalized learning systems, automated assessments, described as the technology that builds systems to think and act facial recognition systems, chatbots (social media sites), and like humans with the ability of achieving goals. AI is mainly predictive analytics tools are being deployed increasingly known through different applications and advanced computer in K-12 educational settings; they are powered by machine- programs, such as recommender systems (e.g., YouTube, Net- learning systems and algorithms. These applications flix), personal assistants (e.g., Apple’s Siri), facial recognition of AI have shown promise to support teachers and students systems (e.g., Facebook’s face detection in photographs), and in various ways: (a) providing instruction in mixed-ability learning apps (e.g., Duolingo). To build on these pro- classrooms, (b) providing students with detailed and timely grams, different sub-fields of AI have been used in a diverse feedback on their writing products, (c) freeing teachers from range of applications. Evolutionary algorithms and machine the burden of possessing all knowledge and giving them learning are most relevant to AI in K-12 education. more room to support their students while they are observ- ing, discussing, and gathering information in their collabo- 2.1 Algorithms rative knowledge-building processes [26, 50]. Below, we outline benefits of each of these educational applications in Algorithms are the core elements of AI. The history of AI the K-12 setting before turning to a synthesis of their ethical is closely connected to the development of sophisticated and challenges and drawbacks. evolutionary algorithms. An algorithm is a set of rules or instructions that is to be followed by computers in problem- solving operations to achieve an intended end goal. In essence, 3.1 Personalized learning systems all computer programs are algorithms. They involve thousands of lines of codes which represent mathematical instructions Personalized learning systems, also known as adaptive learn- that the computer follows to solve the intended problems (e.g., ing platforms or intelligent tutoring systems, are one of the as computing numerical calculation, processing an image, and most common and valuable applications of AI to support grammar-checking in an essay). AI algorithms are applied to students and teachers. They provide students access to dif- fields that we might think of as essentially human behavior— ferent learning materials based on their individual learning such as speech and face recognition, visual perception, learn- needs and subjects. For example, rather than practicing ing, and decision-making and learning. In that way, algorithms chemistry on a worksheet or reading a textbook, students can provide instructions for almost any AI system and applica- may use an adaptive and interactive multimedia version tion we can conceive. of the course content. Comparing students’ scores on researcher-developed or standardized tests, research shows 2.2 Machine learning that the instruction based on personalized learning systems resulted in higher test scores than traditional teacher-led Machine learning is derived from statistical learning methods instruction. Microsoft’s recent report (2018) of over and uses data and algorithms to perform tasks which are typi- 2000 students and teachers from Singapore, the U.S., the cally performed by humans. Machine learning is about UK, and Canada shows that AI supports students’ learning making computers act or perform without being given any progressions. These platforms promise to identify gaps in line-by-line step. The working mechanism of machine students’ prior knowledge by accommodating learning tools learning is the learning model’s exposure to ample amounts and materials to support students’ growth. These systems of quality data. Machine-learning algorithms first analyze generate models of learners using their knowledge and cog- the data to determine patterns and to build a model and then nition; however, the existing platforms do not yet provide predict future values through these models. In other words, models for learners’ social, emotional, and motivational machine learning can be considered a three-step process. First, states. Considering the shift to remote K-12 education it analyzes and gathers the data, and then, it builds a model to during the COVID-19 pandemic, personalized learning sys- excel for different tasks, and finally, it undertakes the action tems offer a promising form of distance learning that could and produces the desired results successfully without human reshape K-12 instruction for the future. intervention [29, 56]. The widely known AI applications such as recommender or facial recognition systems have all been made possible through the working principles of machine 3.2 Automated assessment systems learning. Automated assessment systems are becoming one of the most prominent and promising applications of machine 13 434 AI and Ethics (2022) 2:431–440 learning in K-12 education. These scoring algorithm point out that the integration of social media can foster stu- systems are being developed to meet the need for scoring dents’ active learning, collaboration skills, and connections students’ writing, exams and assignments, and tasks usually with communities beyond the classroom. Chatbots also performed by the teacher. Assessment algorithms can pro- take place in social media outlets through different AI sys- vide course support and management tools to lessen teach- tems. They are also known as dialogue systems or con- ers’ workload, as well as extend their capacity and produc- versational agents [26, 52]. Chatbots are helpful in terms of tivity. Ideally, these systems can provide levels of support their ability to respond naturally with a conversational tone. to students, as their essays can be graded quickly. Pro- For instance, a text-based chatbot system called “Pounce” viders of the biggest open online courses such as Coursera was used at Georgia State University to help students and EdX have integrated automated scoring engines into through the registration and admission process, as well as their learning platforms to assess the writings of hundreds financial aid and other administrative tasks. of students. On the other hand, a tool called “Grad- escope” has been used by over 500 universities to develop In summary, applications of AI can positively impact and streamline scoring and assessment. By flagging students’ and teachers’ educational experiences and help the wrong answers and marking the correct ones, the tool them address instructional challenges and concerns. On the supports instructors by eliminating their manual grading other hand, AI cannot be a substitute for human interaction time and effort. Thus, automated assessment systems deal [22, 47]. Students have a wide range of learning styles and very differently with marking and giving feedback to essays needs. Although AI can be a time-saving and cognitive aide compared to numeric assessments which analyze right or for teachers, it is but one tool in the teachers’ toolkit. There- wrong answers on the test. Overall, these scoring systems fore, it is critical for teachers and students to understand the have the potential to deal with the complexities of the teach- limits, potential risks, and ethical drawbacks of AI applica- ing context and support students’ learning process by provid- tions in education if they are to reap the benefits of AI and ing them with feedback and guidance to improve and revise minimize the costs. their writing. 3.3 Facial recognition systems and predictive 4 Ethical concerns and potential risks of AI analytics applications in education Facial recognition software is used to capture and moni- The ethical challenges and risks posed by AI systems seem- tor students’ facial expressions. These systems provide ingly run counter to marketing efforts that present algorithms insights about students’ behaviors during learning processes to the public as if they are objective and value-neutral tools. and allow teachers to take action or intervene, which, in In essence, algorithms reflect the values of their builders turn, helps teachers develop learner-centered practices and who hold positions of power. Whenever people cre- increase student’s engagement. Predictive analytics ate algorithms, they also create a set of data that represent algorithm systems are mainly used to identify and detect society’s historical and systemic biases, which ultimately patterns about learners based on statistical analysis. For transform into algorithmic bias. Even though the bias is example, these analytics can be used to detect university stu- embedded into the algorithmic model with no explicit inten- dents who are at risk of failing or not completing a course. tion, we can see various gender and racial biases in different Through these identifications, instructors can intervene and AI-based platforms. get students the help they need. Considering the different forms of bias and ethical chal- lenges of AI applications in K-12 settings, we will focus 3.4 Social networking sites and chatbots on problems of privacy, surveillance, autonomy, bias, and discrimination (see Fig. 1). However, it is important to Social networking sites (SNSs) connect students and teach- acknowledge that educators will have different ethical con- ers through social media outlets. Researchers have empha- cerns and challenges depending on their students’ grade sized the importance of using SNSs (such as Facebook) and age of development. Where strategies and resources to expand learning opportunities beyond the classroom, are recommended, we indicate the age and/or grade level of monitor students’ well-being, and deepen student–teacher student(s) they are targeting (Fig. 2). relations. Different scholars have examined the role of One of the biggest ethical issues surrounding the use social media in education, describing its impact on student of AI in K-12 education relates to the privacy concerns and teacher learning and scholarly communication. They 13 AI and Ethics (2022) 2:431–440 435 undermines human agency and privacy. In other words, people’s agency diminishes as AI systems reduce intro- spective and independent thought. Relatedly, schol- ars have raised the ethical issue of forcing students and parents to use these algorithms as part of their education even if they explicitly agree to give up privacy [14, 48]. They really have no choice if these systems are required by public schools. Another ethical concern surrounding the use of AI in K-12 education is surveillance or tracking systems which gather detailed information about the actions and prefer- ences of students and teachers. Through algorithms and machine-learning models, AI tracking systems not only necessitate monitoring of activities but also determine the future preferences and actions of their users. Sur- veillance mechanisms can be embedded into AI’s predic- tive systems to foresee students’ learning performances, strengths, weaknesses, and learning patterns. For instance, Fig. 1  Potential ethical and societal risks of AI applications in educa- research suggests that teachers who use social networking tion sites (SNSs) for pedagogical purposes encounter a number of problems, such as concerns in relation to boundaries of privacy, friendship authority, as well as responsibility and availability. While monitoring and patrolling students’ actions might be considered part of a teacher’s responsibil- ity and a pedagogical tool to intervene in dangerous online cases (such as cyber-bullying or exposure to sexual content), such actions can also be seen as surveillance systems which are problematic in terms of threatening students’ privacy. Monitoring and tracking students’ online conversations and actions also may limit their participation in the learning event and make them feel unsafe to take ownership for their ideas. How can students feel secure and safe, if they know that AI systems are used for surveilling and policing their thoughts and actions?. Problems also emerge when surveillance systems trigger issues related to autonomy, more specifically, the person’s Fig. 2  Student work from the activity of “Youtube Redesign” (MIT ability to act on her or his own interest and values. Predic- Media Lab, AI and Ethics Curriculum, p.1, ) tive systems which are powered by algorithms jeopardize students and teachers’ autonomy and their ability to govern of students and teachers [47, 49, 54]. Privacy violations their own life [46, 47]. Use of algorithms to make predic- mainly occur as people expose an excessive amount of per- tions about individuals’ actions based on their information sonal information in online platforms. Although existing raise questions about fairness and self-freedom. There- legislation and standards exist to protect sensitive personal fore, the risks of predictive analysis also include the perpetu- data, AI-based tech companies’ violations with respect to ation of existing bias and prejudices of social discrimination data access and security increase people’s privacy con- and stratification. cerns [42, 54]. To address these concerns, AI systems ask Finally, bias and discrimination are critical concerns for users’ consent to access their personal data. Although in debates of AI ethics in K-12 education. In AI plat- consent requests are designed to be protective measures forms, the existing power structures and biases are embed- and to help alleviate privacy concerns, many individu- ded into machine-learning models. Gender bias is one als give their consent without knowing or considering of the most apparent forms of this problem, as the bias is the extent of the information (metadata) they are sharing, revealed when students in language learning courses use such as the language spoken, racial identity, biographical AI to translate between a gender-specific language and data, and location. Such uninformed sharing in effect one that is less-so. For example, while Google Translate 13 436 AI and Ethics (2022) 2:431–440 translated the Turkish equivalent of “She/he is a nurse” interaction, see: (a) The Chinese University of Hong Kong into the feminine form, it also translated the Turkish equiv- (CUHK)’s AI for the Future Project (AI4Future) ; (b) alent of “She/he is a doctor” into the masculine form. IBM’s Educator’s AI Classroom Kit , Google’s Teach- This shows how AI models in language translation carry able Machine , UK-based nonprofit organization Apps the societal biases and gender-specific stereotypes in the for Good , and Machine Learning for Kids. data. Similarly, a number of problematic cases of racial bias are also associated with AI’s facial recognition 5.1 "AI and Ethics Curriulum" for middle school systems. Research shows that facial recognition software students by MIT Media Lab has improperly misidentified a number of African Ameri- can and Latino American people as convicted felons. The MIT Media Lab team offers an open-access curricu- Additionally, biased decision-making algorithms reveal lum on AI and ethics for middle school students and teach- themselves throughout AI applications in K-12 education: ers. Through a series of lesson plans and hand-on activi- personalized learning, automated assessment, SNSs, and ties, teachers are guided to support students’ learning of the predictive systems in education. Although the main prom- technical terminology of AI systems as well as the ethical ise of machine-learning models is increased accuracy and and societal implications of AI. The curriculum includes objectivity, current incidents have revealed the contrary. various lessons tied to learning objectives. One of the main For instance, England’s A-level and GCSE secondary learning goals is to introduce students to basic components level examinations were cancelled due to the pandemic of AI through algorithms, datasets, and supervised machine- in the summer of 2020 [1, 57]. An alternative assessment learning systems all while underlining the problem of algo- method was implemented to determine the qualification rithmic bias. For instance, in the activity “AI Bingo”, grades of students. The grade standardization algorithm students are given bingo cards with various AI systems, such was produced by the regulator Ofqual. With the assess- as online search engine, customer service bot, and weather ment of Ofqual’s algorithm based on schools' previous app. Students work with their partners collaboratively on examination results, thousands of students were shocked to these AI systems. In their AI Bingo chart, students try to receive unexpectedly low grades. Although a full discus- identify what prediction the selected AI system makes and sion of the incident is beyond the scope of this article what dataset it uses. In that way, they become more familiar it revealed how the score distribution favored students who with the notions of dataset and prediction in the context of attended private or independent schools, while students AI systems. from underrepresented groups were hit hardest. Unfortu- In the second investigation, “Algorithms as Opinions”, nately, automated assessment algorithms have the potential students think about algorithms as recipes, which are cre- to reconstruct unfair and inconsistent results by disrupting ated by set of instructions that modify an input to produce student’s final scores and future careers. an output. Initially, students are asked to write an algo- rithm to make the “best” jelly sandwich and peanut but- ter. They explore what it means to be “best” and see how their opinions of best in their recipes are reflected in their 5 Teaching and understanding AI and ethics algorithms. In this way, students are able to figure out that in educational settings algorithms can have various motives and goals. Following this activity, students work on the “Ethical Matrix”, build- These ethical concerns suggest an urgent need to introduce ing on the idea of the algorithms as opinions. During students and teachers to the ethical challenges surround- this investigation, students first refer back to their developed ing AI applications in K-12 education and how to navi- algorithms through their “best” jelly sandwich and peanut gate them. To meet this need, different research groups butter. They discuss what counts as the “best” sandwich for and nonprofit organizations offer a number of open-access themselves (most healthy, practical, delicious, etc.). Then, resources based on AI and ethics. They provide instruc- through their ethical matrix (chart), students identify differ- tional materials for students and teachers, such as lesson ent stakeholders (such as their parents, teacher, or doctor) plans and hands-on activities, and professional learning who care about their peanut butter and jelly sandwich algo- materials for educators, such as open virtual learning ses- rithm. In this way, the values and opinions of those stake- sions. Below, we describe and evaluate three resources: holders also are embedded in the algorithm. Students fill out “AI and Ethics” curriculum and “AI and Data Privacy” an ethical matrix and look for where those values conflict workshop from the Massachusetts Institute of Technology or overlap with each other. This matrix is a great tool for (MIT) Media Lab as well as Code.org’s “AI and Oceans” students to recognize different stakeholders in a system or activity. For readers who seek to investigate additional society and how they are able to build and utilize the values approaches and resources for K-12 level AI and ethics of the algorithms in an ethical matrix. 13 AI and Ethics (2022) 2:431–440 437 The final investigation which teaches about the biased open-access online document, freely available to teachers nature of algorithms is “Learning and Algorithmic Bias”.. During the investigation, students think further about The first workshop in the series is “Mystery YouTube the concept of classification. Using Google’s Teachable Viewer: A lesson on Data Privacy”. During the workshop, Machine tool , students explore the supervised machine- students engage with the question of what privacy and data learning systems. Students train a cat–dog classifier using mean. They observe YouTube’s home page from the two different datasets. While the first dataset reflects the cats perspective of a mystery user. Using the clues from the vid- as the over-represented group, the second dataset indicates eos, students make predictions about what the characters the equal and diverse representation between dogs and cats in the videos might look like or where they might live. In a. Using these datasets, students compare the accuracy way, students imitate YouTube algorithms’ prediction mode between the classifiers and then discuss which dataset and about the characters. Engaging with these questions and outcome are fairer. This activity leads students into a dis- observations, students think further about why privacy and cussion about the occurrence of bias in facial recognition boundaries are important and how each algorithm will inter- algorithms and systems. pret us differently based on who creates the algorithm itself. In the rest of the curriculum, similar to the AI Bingo The second workshop in the series is “Designing ads with investigation, students work with their partners to determine transparency: A creative workshop”. Through this work- the various forms of AI systems in the YouTube platform shop, students are able to think further about the meaning, (such as its recommender algorithm and advertisement aim, and impact of advertising and the role of advertise- matching algorithm). Through the investigation of “YouTube ments in our lives. Students collaboratively create an Redesign”, students redesign YouTube’s recommender sys- advertisement using an everyday object. The objective is to tem. They first identify stakeholders and their values in the make the advertisement as “transparent” as possible. To do system, and then use an ethical matrix to reflect on the goals that, students learn about notions of malware and adware, as of their YouTube’s recommendation algorithm. Finally, well as the components of YouTube advertisements (such as through the activity of “YouTube Socratic Seminar”, stu- sponsored labels, logos, news sections, etc.). By the end of dents read an abridged version of Wall Street Journal article the workshop, students design their ads as a poster, and they by participating in a Socratic seminar. The article was edited share with their peers. to shorten the text and to provide more accessible language The final workshop in MIT’s AI and data privacy series for middle school students. They discuss which stakeholders is “Designing Privacy in Social Media Platforms”. This were most influential or significant in proposing changes in workshop is designed to teach students about YouTube, the YouTube Kids app and whether or not technologies like design, civics, and data privacy. During the workshop, auto play should ever exist. During their discussion, students students create their own designs to solve one of the big- engage with the questions of: “Which stakeholder is making gest challenges of the digital era: problems associated with the most change or has the most power?”, “Have you ever online consent. The workshop allows students to learn more seen an inappropriate piece of content on YouTube? What about the privacy laws and how they impact youth in terms did you do?”. of media consumption. Students consider YouTube within Overall, the MIT Media Lab’s AI and Ethics curriculum the lenses of the Children’s Online Privacy Protections Rule is a high quality, open-access resource with which teachers (COPPA). In this way, students reflect on one of the com- can introduce middle school students to the risks and ethi- ponents of the legislation: how might students get parental cal implications of AI systems. The investigations described permission (or verifiable consent)? above involve students in collaborative, critical thinking Such workshop resources seem promising in helping activities that force them to wrestle with issues of bias and educate students and teachers about the ethical challenges discrimination in AI, as well as surveillance and autonomy of AI in education. Specifically, social media such as You- through the predictive systems and algorithmic bias. Tube are widely used as a teaching and learning tool within K-12 classrooms and beyond them, in students’ everyday 5.2 “AI and Data Privacy” workshop series for K‑9 lives. These workshop resources may facilitate teachers’ students by MIT Media Lab and students’ knowledge of data privacy issues and support them in thinking further about how to protect privacy online. Another quality resource from the MIT Media Lab’s Per- Moreover, educators seeking to implement such resources sonal Robots Group is a workshop series designed to teach should consider engaging students in the larger question: students (between the ages 7 and 14) about data privacy who should own one’s data? Teaching students the underly- and introduce them to designing and prototyping data pri- ing reasons for laws and facilitating debate on the extent to vacy features. The group has made the content, materials, which they are just or not could help get at this question. worksheets, and activities of the workshop series into an 13 438 AI and Ethics (2022) 2:431–440 5.3 Investigation of “AI for Oceans” by Code.org responsive pedagogies (by focusing on students’ funds of knowledge, family background, and cultural experiences) A third recommended resource for K-12 educators trying while creating instructional materials that address surveil- to navigate the ethical challenges of AI with their students lance, privacy, autonomy, and bias. In such student-centered comes from Code.org, a nonprofit organization focused learning environments, students voice their own cultural and on expanding students’ participation in computer science. contextual experiences while trying to critique and disrupt Sponsored by Microsoft, Facebook, Amazon, Google, and existing power structures and cultivate their social aware- other tech companies, Code.org aims to provide opportu- ness [24, 36]. nities for K-12 students to learn about AI and machine- Finally, as scholars in teacher education and educational learning systems. To support students (grades 3–12) in technology, we believe that educating future generations of learning about AI, algorithms, machine learning, and bias, diverse citizens to participate in the ethical use and devel- the organization offers an activity called “AI for Oceans”, opment of AI will require more professional development where students are able to train their machine-learning for K-12 teachers (both pre-service and in-service). For models. instance, through sustained professional learning sessions, The activity is provided as an open-access tutorial for teachers could engage with suggested curriculum resources teachers to help their students explore how to train, model and teaching strategies as well as build a community of and classify data, as well as to understand how human bias practice where they can share and critically reflect on their plays a role in machine-learning systems. During the activ- experiences with other teachers. Further research on such ity, students first classify the objects as either “fish” or “not reflective teaching practices and students’ sense-making pro- fish” in an attempt to remove trash from the ocean. Then, cesses in relation to AI and ethics lessons will be essential to they expand their training data set by including other sea developing curriculum materials and pedagogies relevant to creatures that belong underwater. Throughout the activity, a broad base of educators and students. students are also able to watch and interact with a number of visuals and video tutorials. With the support of their teach- ers, they discuss machine learning, steps and influences of Funding This work was supported by the Graduate School at Michigan State University, College of Education Summer Research Fellowship. training data, as well as the formation and risks of biased data. Data availability Not applicable. Code availability Not applicable. 6 Future directions for research and teaching on AI and ethics Declarations In this paper, we provided an overview of the possibilities Conflict of interest The authors declare that they have no conflict of and potential ethical and societal risks of AI integration in interest. education. To help address these risks, we highlighted sev- eral instructional strategies and resources for practitioners seeking to integrate AI applications in K-12 education and/ References or instruct students about the ethical issues they pose. These instructional materials have the potential to help students 1. Adams, R., McIntyre, N.: England A-level downgrades hit pupils from disadvantaged areas hardest.https://​www.​thegu​ardian.​com/​ and teachers reap the powerful benefits of AI while navi- educa​tion/​2020/​aug/​13/​engla​nd-a-​level-​downg ​rades-​hit-​pupils-​ gating ethical challenges especially related to privacy con- from-d​ isadv​ antag​ ed-a​ reas-h​ ardes​ t (2020). Accessed 10 September cerns and bias. Existing research on AI in education provides 2020 insight on supporting students’ understanding and use of AI 2. Ali, S. A., Payne, B. H., Williams, R., Park, H. W., Breazeal, C.: Constructionism, ethics, and creativity: developing primary and [2, 13]; however, research on how to develop K-12 teachers’ middle school artificial intelligence education. Paper presented instructional practices regarding AI and ethics is still in its at International Workshop on Education in Artificial Intelligence infancy. (EDUAI). Palo Alto, CA, USA. (2019) Moreover, current resources, as demonstrated above, 3. Almeida, D., Shmarko, K., Lomas, E.: The ethics of facial recog- nition technologies, surveillance, and accountability in an age of mainly address privacy and bias-related ethical and soci- artificial intelligence: a comparative analysis of US, EU, and UK etal concerns of AI. Conducting more exploratory and regulatory frameworks. AI Ethics (2021). https://d​ oi.o​ rg/1​ 0.1​ 007/​ critical research on teachers’ and students’ surveillance and s43681-​021-​00077-w autonomy concerns will be important to designing future 4. Apps for Good: https://​www.​appsf​orgood.​org/​about Accessed 28 August 2021 resources. In addition, curriculum developers and workshop designers might consider centering culturally relevant and 13 AI and Ethics (2022) 2:431–440 439 5. Asterhan, C.S.C., Rosenberg, H.: The promise, reality and dilem- 27. Henderson, P., Sinha, K., Angelard-Gontier, N., Ke, N. R., Fried, mas of secondary school teacher–student interactions in Face- G., Lowe, R., Pineau, J.: Ethical challenges in data-driven dia- book: the teacher perspective. Comput. Educ. 85, 134–148 (2015) logue systems. In: Proceedings of AAAI/ACM Conference on 6. Krutka, D., Manca, S., Galvin, S., Greenhow, C., Koehler, M., AI Ethics and Society (AIES-18), New Orleans, Lousiana, USA. Askari, E.: Teaching “against” social media: confronting problems (2000) of profit in the curriculum. Teachers College Record 121(14), 28. Herder, E., Sosnovsky, S., Dimitrova, V.: Adaptive intelligent 1–42 (2019) learning environments. In: Duval, Erik, Sharples, Mike, Suther- 7. Greenhow, C., Galvin, S., Brandon, D., Askari, E.: A decade of land, Rosamund (eds.) Technology enhanced learning, pp. 109– research on K-12 teaching and teacher learning with social media: 114. Springer International Publishing, Cham (2017) insights on the state-of-the-field. Teachers College Record 122(6), 29. Holmes, W., Bialik, M., Fadel, C.: Artificial intelligence in educa- 1–7 (2020) tion: promises and implications for teaching and learning. Center 8. Greenhow, C., Galvin, S., Staudt Willet, K.B.: Inquiring tweets for Curriculum Redesign, Boston (2019) want to know: #Edchat supports for #remoteteaching during 30. IBM the Educator's AI in the Classroom Toolkit: https://​docs.​ COVID-19. British Journal of Educational Technology. 1–21, google. ​ c om/ ​ d ocum ​ e nt/d/ ​ 1 Zqi7 ​ 4 ejYY ​ Y LAAE ​ j FBJI ​ YuXUc​ (2021) OMQI5​6R6K0​ZsvQb​DmOA/​edit Accessed 28 August 2021 9. Barr, A., Feigenbaum, E.A.: Handbook of artificial intelligence, 31. IEEE Corporate Advisory Group (CAG): IEEE guide for terms vol. 1and 2. Kaufmann, Los Altos (1981) and concepts in intelligent process automation. The Institute of 10. Bendici, R.: Rise of the machines. Univ. Bus. 21(10), 53–54 Electrical and Electronics Engineers Standards Association. 1–16 (2018) (2017). https://​ieeex​plore.​ieee.​org/​iel7/​80706​69/​80706​70/​08070​ 11. Blank, G., Bolsover, G., Dubois, E.: A new privacy paradox: 671.​pdf young people and privacy on social network site. Oxford Inter- 32. Iman, M., Arabnia, H. R., Branchinst, R. M.: Pathways to artificial net Institute (2014) general intelligence: a brief overview of developments and ethical 12. Blumenstyk, G.: Can artificial intelligence make teaching more issues via artificial intelligence, machine learning, deep learning, personal? The Chronicle of Higher Education. https://​w ww.​ and data science. Paper presented at International Conference on chron​icle.​com/​artic​le/​Can-​Artif​i cial-​Intel​ligen​ce/​243023 (2018) Artificial Intelligence (ICAI). Las Vegas, Nevada, USA. (2020) 13. Borenstein, J., Howard, A.: Emerging challenges in AI and the 33. Johnson, M.: A scalable approach to reducing gender bias in need for AI ethics education. AI Ethics 1, 61–65 (2020) Google translate. https://​ai.​googl​eblog.​com/​2020/​04/a-​scala​ble-​ 14. Bulger, M.: Personalized learning: the conversations we’re not appro​ach-​to-​reduc​ing-​gender.​html Accessed 26 March 2021 having. Data and Society Research Institute. https://​datas​ociety.​ 34. Ko, A.J., Oleson, A., Ryan, N., Register, Y., Xie, B., Tari, M., net/l​ ibrar​ y/p​ erson​ alize​ d-l​ earni​ ng-t​ he-c​ onver​ satio​ ns-w ​ ere-n​ ot-h​ av- Davidson, M., Druga, S., Loksa, D.: It is time for more critical ing/ (2016) CS education. Commun. ACM 63(11), 31–33 (2020) 15. Cath, C., Wachter, S., Mittelstadt, B., Tadder, M., Floridi, L.: 35. Krueger, N.: Artificial intelligence has infiltrated our lives. Can it Artificial Intelligence and the ‘good society’: the US, EU, and improve learning? International Society for Technology in Educa- UK approach. Sci. Eng. Ethics 24, 505–528 (2018) tion (ISTE). (2017). 16. Chaudhry, M.A., Kazim, E.: Artificial intelligence in education 36. Ladson-Billings, G.: Toward a theory of culturally relevant peda- (AIEd): a high-level academic and industry note 2021. AI Ethics gogy. Am. Educ. Res. J. 32(3), 465–491 (1995) (2021). https://​doi.​org/​10.​1007/​s43681-​021-​00074-z 37. Machine Learning for Kids:https://​machi​nelea​r ning​forki​ds.​co.​ 17. Cheney-Lippold, J.: We are data: algorithms and the making of uk/#​!/​links. Accessed 28 August 2021 our digital selves. New York University Press, New York (2017) 38. McCarthy, J.: What is artificial intelligence? http://​jmc.​stanf​ord.​ 18. Chiu, T.K.F., Meng, H., Chai, C.S., King, I., Wong, S., Yam, Y.: edu/​artif​i cial-​intel​ligen​ce/​what-​is-​ai/ Accessed 28 August 2021 Creation and evaluation of a pre-tertiary artificial intelligence (AI) 39. McMurtrie, B.: How artificial intelligence is changing teaching. curriculum. IEEE Trans. Educ. (2021). https://​doi.​org/​10.​1109/​ The Chronicle of Higher Education.https://​www.​chron​icle.​com/​ TE.​2021.​30858​78 artic​le/​How-​Artif​i cial-​Intel​ligen​ce-​Is/​244231 (2018) 19. Citron, D.K., Pasquale, F.A.: The scored society: due process for 40. Miller, F.A., Katz, J.H., Gans, R.: The OD imperative to add inclu- automated predictions. Wash. Law Rev. 89, 1–33 (2014) sion to the algorithms of artificial intelligence. OD Practitioner. 20. Code.org.: AI for Oceans. https://​code.​org/​oceans (2020) 5(1), 6–12 (2018) 21. Dignum, V.: Ethics in artificial intelligence: introduction to the 41. Mohri, M., Rostamizadeh, A., Talwalkar, A.: Foundations of special issue. Ethics Inf. Technol. 20, 1–3 (2018) machine learning. MIT Press, Cambridge (2012) 22. Dishon, G.: New data, old tensions: big data, personalized learn- 42. Murphy, R. F.: Artificial intelligence applications to support k–12 ing, and the challenges of progressive education. Theory Res. teachers and teaching: a review of promising applications, chal- Educ. 15(3), 272–289 (2017) lenges, and risks. Perspective. 1–20 (2019). https://​doi.​org/​10.​ 23. Tiple, Vasile, Recommendations on the European Commission’s 7249/​PE315 WHITE PAPER on Artificial Intelligence - A European approach 43. Naqvi, A.: Artificial intelligence for audit, forensic accounting, to excellence and trust, COM(2020) 65 final (the 'AI White Paper') and valuation: a strategic perspective. Wiley (2020) (2020). https://doi.org/10.2139/ssrn.3706099 44. Nguyen, S., DiPaola, D.: AI & data privacy activities for k-9 24. Gay, G.: Culturally responsive teaching: theory, research, and students. MIT Media Lab. https://​www.​media.​mit.​edu/​proje​cts/​ practice. Teachers College Press, New York (2010) data-​priva​cy-​design-​for-​youth/​overv​iew/ (2020) 25. Google Teachable Machine: https://t​ eacha​ blema​ chine.w ​ ithgo​ ogle.​ 45. Payne, B. H.: An ethics of artificial intelligence curriculum for com/ Accessed 28 August 2021 middle school students. MIT Media Lab. https://​www.​media.​mit.​ 26. Hrastinski, S., Olofsson, A.D., Arkenback, C., Ekström, S., edu/​proje​cts/​ai-​ethics-​for-​middle-​school/​overv​iew/ (2019) Ericsson, E., Fransson, G., Jaldemark, J., Ryberg, T., Öberg, L., 46. Piano, S.L.: Ethical principles in machine learning and artificial Fuentes, A., Gustafsson, U., Humble, N., Mozelius, P., Sundgren, intelligence: cases from the field and possible ways forward. M., Utterberg, M.: Critical imaginaries and reflections on artificial Humanit. Soc. Sci. Commun. 7(9), 1–7 (2020) intelligence and robots in postdigital K-12 education. Postdigit. Sci. Educ. 1, 427–445 (2019) 13 440 AI and Ethics (2022) 2:431–440 47. Regan, P.M., Jesse, J.: Ethical challenges of edtech, big data and 55. The Institute for Ethical AI in Education. Interim report: towards a personalized learning: twenty-first century student sorting and shared vision of ethical AI in education. https://w ​ ww.b​ uckin​ gham.​ tracking. Ethics Inf. Technol. 21, 167–179 (2019) ac.​uk/​wp-​conte​nt/​uploa​ds/​2020/​02/​The-​Insti​tute-​for-​Ethic​al-​AI-​ 48. Regan, P. M., Steeves, V.: Education, privacy, and big data algo- in-​Educa​tions-​Inter​im-​Report-​Towar​ds-a-​Shared-​Vision-​of-​Ethic​ rithms: taking the persons out of personalized learning. First Mon- al-​AI-​in-​Educa​tion.​pdf. (2020) day, 24(11), (2019) 56. Waller, M., Paul, W.: Why predictive algorithms are so risky for 49. Remian, D.: Augmenting education: ethical considerations for public sector bodies. Social Science Research Network, Rochester incorporating artificial intelligence in education (Unpublished (2020) master’s thesis). University of Massachusetts, Boston (2019) 57. Weale, S., Stewart, H.: A-level and GCSE results in England to be 50. Roll, I., Wylie, R.: Evolution and revolution in artificial intelli- based on teacher assessments in U-turn. The Guardian, (2020). gence in education. Int. J. Artif. Intell. Educ. 26, 582–599 (2016) 58. Zimmerman, M.: Teaching AI: exploring new frontiers for learn- 51. Smith, H.: Algorithmic bias: should students pay the price? AI ing. International Society for Technology in Education, Portland Soc. 35, 1077–1078 (2020) (2018) 52. Smutny, P., Schreiberova, P.: Chatbots for learning: a review of educational chatbots for the Facebook Messenger. Comput. Educ. Publisher's Note Springer Nature remains neutral with regard to 151, 1–11 (2020) jurisdictional claims in published maps and institutional affiliations. 53. Specia, M.: Parents, Students and Teachers Give Britain a Failing Grade Over Exam Results. The New York Times, (2020). https://​ www.​nytim​es.​com/​2020/​08/​14/​world/​europe/​engla​nd-a-​level-​ resul​ts.​html 54. Stahl, B.C., Wright, D.: Ethics and privacy in ai and big data: implementing responsible research and innovation. IEEE Secur. Priv. 16(3), 26–33 (2018) 13