Machine Learning and the Five Big Ideas in AI_removed.pdf

International Journal of Artificial Intelligence in Education (2023) 33:233–266 https://doi.org/10.1007/s40593-022-00314-1 ARTICLE Machine Learning and the Five Big Ideas in AI David Touretzky1 · Christina Gardner‑McCune2 · Deborah Seehorn3 Accepted: 23 September 2022 / Published online: 27 October 2022 © International Artificial Intelligence in Education Society 2022 Abstract This article provides an in-depth look at how K-12 students should be introduced to Machine Learning and the knowledge and skills they will develop as a result. We begin with an overview of the AI4K12 Initiative, which is developing national guidelines for teaching AI in K-12, and briefly discuss each of the “Five Big Ideas in AI” that serve as the organizing framework for the guidelines. We then discuss the general format and structure of the guidelines and grade band progression charts and provide a theoretical framework that highlights the developmental appropriateness of the knowledge and skills we want to impart to students and the learning expe- riences we expect them to engage in. Development of the guidelines is informed by best practices from Learning Sciences and CS Education research, and by the need for alignment with CSTA’s K-12 Computer Science Standards, Common Core standards, and Next Generation Science Standards (NGSS). The remainder of the article provides an in-depth exploration of the AI4K12 Big Idea 3 (Learning) grade band progression chart to unpack the concepts we expect students to master at each grade band. We present examples to illustrate the progressions from two perspec- tives: horizontal (across grade bands) and vertical (across concepts for a given grade band). Finally, we discuss how these guidelines can be used to create learning expe- riences that make connections across the Five Big Ideas, and free online tools that YENDEN facilitate these experiences. Keywords Artificial intelligence · Machine learning · AI4K12 Initiative · Five Big Ideas in AI · Reasoning models · Grade band progression chart · K-12 AI education * David Touretzky [email protected] Christina Gardner‑McCune [email protected] Deborah Seehorn [email protected] 1 Carnegie Mellon University, Pittsburgh, PA, USA 2 University of Florida, Gainesville, FL, USA 3 CS4NC / NC ECEP, Wilmington, NC, USA 13 Vol.:(0123456789) 234 International Journal of Artificial Intelligence in Education (2023) 33:233–266 Introduction Children today grow up surrounded by AI technologies including speech and facial recognition systems, intelligent assistants such as Alexa and Siri, recom- mender systems for products, news stories, and social media feeds, and nearly- autonomous automobiles with extensive driver assistance features. If Artificial Intelligence is truly “the new electricity” (Andrew Ng, quoted in Jewell, 2019) and a key component of the “fourth industrial revolution” (Qiang & Chao, 2018), it is imperative that children understand it (Hasse et al., 2019; Druga et al., 2017). Learning about AI has the potential to improve their critical thinking skills and increase their confidence in interacting with computing systems. Educating chil- dren about AI is also important for civic reasons (Ali et al., 2021). Disruptive AI technologies such as opaque and possibly biased automated decision-making sys- tems (Buolamwini & Gebru, 2018; Kirkpatrick, 2016; Selbst, 2017a, 2017b; Van Brakel, 2016), ubiquitous surveillance capabilities (McStay, 2020; Shachar et al., 2020), and realistic deepfakes raise ethical issues (Diakopoulos & Johnson, 2021; Rini, 2020) that should be confronted by a well-informed citizenry (Lao, 2020). K-12 AI education is also needed to develop our future workforce (Zhang et al., 2022). Companies recognize AI and specifically machine learning as inte- gral to their ability to stay competitive, although they are still evaluating what AI can do for their business (Deloitte, 2020; Gartner, 2020). AI is poised to cause major shifts in the types of skills workers will need. Although some jobs will disappear completely due to automation, most will evolve to integrate AI into their daily workflow. New job opportunities will also arise to develop AI- powered solutions to business problems and drive new innovations (Gartner, 2020). While specific job descriptions and titles for AI-enabled careers are yet to be determined, it is certain that future workers will understand the founda- tional building blocks of AI and be able to combine capabilities such as pattern recognition, prediction, image classification, cognitive search, and natural lan- guage understanding to solve practical problems (Deloitte, 2020; Department of Defense, 2018, 2019; SAS Institute, 2018). Since 2018 the AI4K12 Initiative has been developing national guidelines for teaching AI in K-12 (Touretzky et al., 2019a). We hope to give students more than a superficial knowledge of AI terminology and applications. The guidelines provide four levels of engagement: (1) awareness of AI in students’ everyday lives and understanding of its societal impacts; (2) conceptual understanding of how AI works from a systems level perspective and at an algorithmic level; (3) the ethical and responsible design of automated decision-making systems using AI; and (4) skills for applying AI to real-world problems. We believe the combi- nation of these four levels of engagement will prepare K-12 students for AI-ena- bled careers of the future that span both technical and non-technical career paths. In addition we hope to foster an appreciation for some of the profound ideas that underlie our current understanding of human intelligence. While our efforts focus on American students, K-12 AI education is becom- ing a priority world-wide. China has mandated that all K-12 stuñdents receive 13 International Journal of Artificial Intelligence in Education (2023) 33:233–266 235 instruction in AI and is pursuing a variety of approaches to realize this goal (Petersen et al., 2021). The first AI textbook specifically for high school students was published in China in 2018 (China Daily, 2018), and was followed by a series of AI books designed for primary and secondary levels. The European Union is developing AI education through its Erasmus+ program, which is focusing on teaching AI in high schools (Universidad da Coruña, 2019). In 2022 UNESCO released a survey of government-endorsed AI curricula that found 11 countries with AI curricula already implemented (Armenia, Austria, Belgium, China, India, Republic of Korea, Kuwait, Portugal, Qatar, Serbia, and United Arab Emirates), and another 5 with government-endorsed curricula in development (Germany, Jordan, Bulgaria, Saudi Arabia, and Serbia). The United States did not appear in this list because the many AI curriculum resources developed there are not government-endorsed (UNESCO, 2022), but the report does reference both Lao’s Machine Learning Education Framework (Lao, 2020) and the AI4K12 guidelines (AI4K12.org). Recognizing the importance of AI education for K-12 students, the question then becomes what exactly do K-12 students need to learn? We drew upon the exper- tise of AI researchers to identify the content that students should learn, and of K-12 teachers to determine what students could learn. The guidelines for teaching machine learning we put forward in this paper also align with research in the learn- ing sciences and computer science education literature on how to effectively pro- mote children’s learning of complex concepts. Our current articulation of the guidelines is a series of grade band progression charts covering Five Big Ideas in AI in grades K-2, 3–5, 6–8, and 9–12. A draft of the chart for Big Idea 3 (Learning) was released for public feedback in November of 2020. In this paper we examine the content progression for Big Idea 3 in detail. We consider: The logic behind our development of the content progression. The developmental appropriateness of the guidelines, aligning them with other skills students are expected to have or develop within their grade band. The essential insights about machine learning, automated decision making, and development of internal representations we hope students will acquire. The kinds of learning experiences we want students to have. The availability of tools to provide these experiences. The relationship of machine learning to the other big ideas in AI. Related Work AI involves many complex concepts such as statistical inference, parameterized rea- soners (e.g., decision trees or neural networks), and algorithms for adjusting those parameters (learning algorithms). Although research on how and what students can learn about AI at different grade levels is still developing, research in the learning sciences and computing education has shown that K-12 students can engage deeply with complex topics. There are three key insights from these disciplines that we 13 236 International Journal of Artificial Intelligence in Education (2023) 33:233–266 think are applicable to AI Education: model building, experimentation, and con- struction of computational artifacts. (1) Model Building. Learning Sciences research has shown that as learners cre- ate or interact with physical or computational models of scientific phenomena, they develop and refine their own mental models of the phenomena (Lehrer & Schauble, 2006; Linn, 2000; Chittleborough & Treagust, 2009). The Learning By Design (LBD) approach (Kolodner et al., 1998) has shown that engaging in design and modeling activities enhances learning about complex systems through systematic exploration. For example, Hmelo et al. (2000) showed that when children design artificial lungs and build partial working models of the respiratory system, they develop an understanding of the system. Similarly, the BodyVis project allows students to design interactive self-sensing wearables that display the dynamic inner-workings of the wearer’s anatomy and allow participants to explore how their emotions affected their physiology (Norooz et al., 2016). These examples encourage us to expect that even young students, when presented with opportunities to interact with AI models and create their own models, will be able to develop understandings (i.e., mental models and explanations) of AI concepts. (2) Experimentation. Research on students’ experimentation with NetLogo’s multi-agent models has shown such experimentation helps students develop deep understandings of complex phenomena (Wilensky & Rand, 2015; Rand & Wilensky, 2008. For example, in Wilensky and Rand (2015) students experi- mented with a series of simulations of the spread of fire through a forest. The simulations allow students to vary the density of trees, probability of patch-to- patch spread, wind direction, and long distance transport of sparks (called “big jumps”). The simulations show that a fire’s chance of reaching the opposite edge of the forest depends critically on the density of trees. One of the models in particular highlights the behavior of a spark as a critical parameter that affects the spread of the fire across unburned forest segments via big jumps. Studies of student’s experimentation with the simulator have shown that it helps them understand relationships between micro-level behavior (e.g., sparks) and the emergent macro-level behavior that results (e.g., spread of a forest fire) (Wilk- erson et al., 2008). More generally, studies of students experimenting with a variety of NetLogo simulations have shown that representing difficult or often misunderstood phenomena in terms of micro-level interactions better equips stu- dents to accurately describe, explain, and even predict the simulation’s behavior. A growing collection of online demonstrations allows students to experiment with AI. Google’s AI Experiments collection (2022) offers fun and engaging experiences with a variety of AI technologies. More serious experimentation tools include visualizations of neural networks, e.g., TensorFlow Playground (Sato, 2016; Smilkov & Carter, 2016), CNN Explainer (Wang et al., 2020, 2021), and FaceDemo (Makwana et al., 2022), exploration of word embeddings (Bandy- opadhyay et al., 2022; Kahn et al., 2022), text generation or question answering demonstrations using transformer networks (InferKit, Inc., 2020; Lane, 2021b), and gridworld reinforcement learning simulators (IMAGINARY gGmbH, 2021; 13 International Journal of Artificial Intelligence in Education (2023) 33:233–266 237 Karpathy, 2015). But few studies have been published to date that explore how experimentation with these tools helps students learn about important AI con- cepts. (3) Construction of computational artifacts. The Constructionist learning approach suggests that people learn particularly well by constructing person- ally meaningful artifacts (Kafai & Resnick, 1996). The work of the MIT Media Lab has highlighted for years that we need to provide students access to compu- tational tools with low floors, high ceilings, and wide walls that allow them to create personally relevant computational artifacts (Resnick & Silverman, 2005). This research has shown that creation of computational artifacts helps students apply the concepts presented to them and is a key driver in their actual learning of the concepts (Papavlasopoulou et al., 2019; Resnick et al., 2000). For example, in MIT App Inventor, students are able to create mobile applications that can be deployed on Android and iOS devices. In the process of developing these applications they learn about event-based programming, user interface design principles, and short-term and long-term data storage (Turbak et al., 2014). These concepts are typically taught to upper level undergraduates due to their complexity, but MIT App Inventor makes use and discussion of these concepts easy and natural (Grover & Pea, 2013; Chatzinikolakis & Papadakis, 2014). Similarly, Scratch can be used to create games and animated stories about top- ics of personal interest. In doing so, students engage in computational thinking (e.g., problem decomposition and algorithm design) and make use of fundamen- tal computing constructs such as loops, variables, conditionals, and functions (Brennan & Resnick, 2012). Through this process they develop an understanding of these concepts that is grounded in personal experience. In summary, when students are provided with tools to create and not just consume technology, they are inspired to express themselves, increase their interest in computing, and develop their computational identity (Pang, 2022). Extending the constructionist principles to AI education, researchers at MIT and other institutions have developed AI extensions to several block-based pro- gramming environments. Examples include AI for MIT App Inventor (Van Brum- melen et al., 2021), Cognimates (Druga, 2018a, b), Scratch Face Sensing Lab (Scratch Team, 2021a, b), and AI blocks for Snap! (Kahn et al., 2020a, b). These extensions allow students to create applications that incorporate AI components such as speech recognition, sentiment analysis, and classifiers trained via machine learning. Calypso (Touretzky, 2017) is another AI programming framework that evolved from a children’s game development language: Microsoft’s Kodu Game Lab (MacLaurin, 2011). Calypso incorporates speech recognition and generation, computer vision, face detection, path planning, and robot control. Use of these AI-enhanced programming frameworks can lead students to see themselves as not just programmers, but AI application developers. In developing the AI4K12 guidelines we’ve taken into account these insights from the learning sciences and incorporated numerous opportunities for students to develop models, experiment with mechanisms, and create computational artifacts. 13 238 International Journal of Artificial Intelligence in Education (2023) 33:233–266 Background on the AI4K12 Guidelines In 2018 at the beginning of our development of the guidelines, we surveyed the EAAI (Educational Advances in Artificial Intelligence) proceedings archives from 2010 to 2018 and found very little had been published on K-12 AI education that provided even rudimentary guidance about what to teach K-12 students (Touretzky et al., 2019c; Heinze et al., 2010). Most of the K-12 AI & ML publications focused on teaching K-12 students robotics and merely mentioned that robotics provided a motivating context to teach AI and CS concepts, without providing explicit exam- ples. This is not surprising given the dominance of LEGO, FIRST Robotics, and VEX robotics competitions, which are typically limited to simple color or light sen- sors and pre-programmed motion sequences even for the autonomous competitions. As a result, these robotics platforms offer students very little exposure to AI, as most of these robots were unable to see or hear (no camera or microphone), and offered little onboard processing power. Similarly our search of the SIGCSE proceedings and other computing education conferences in the ACM Digital Library and IEEE Xplore Digital Library resulted in few references to teaching AI and machine learn- ing at the K-12 or even undergraduate level. Thus, we turned to existing computer science guidelines and leaned on the expertise of K-12 CS educators and university AI faculty to help us assess what students were capable of understanding and the best ways to adapt college-level AI concepts for K-12 students and teachers. Existing Guidelines for K‑12 Computing Education The K-12 Computer Science Framework (2016) and the CSTA K-12 Computer Sci- ence Standards (CSTA, 2017) define what K-12 students should know about com- puter science. They include a wide range of topics organized within five major con- cepts: (1) computing systems, (2) networks and the internet, (3) data and analysis, (4) algorithms and programming, and (5) impacts of computing. When the CSTA K-12 Computer Science Standards were released, only two standards referenced AI (CSTA, 2017). Both were listed in the advanced high school grade band within the algorithms and programming concept (Fig. 1). Standard 3B-AP-08 involves iden- tifying and communicating the use of AI in software and physical systems, while 3B-AP-09 asks students to create an AI game agent. Beyond these two explicit AI standards, there are several other standards that pro- vide hooks for CS teachers to explore AI concepts, such as data collection, infer- ence & models (e.g., 2-DA-08, 1B-DA-07), designing computational artifacts to Fig. 1 The two references to AI in the 2017 CSTA Computer Science Standards 13 International Journal of Artificial Intelligence in Education (2023) 33:233–266 239 solve societal problems (e.g., 3A-AP-16), evaluation of artifacts and their impacts (e.g., 3A-IC-29), and discussion of bias in the design of computing systems (e.g., 2-IC-21 and 3A-IC-25). See Table 1 for expanded descriptions of these examples. In response to the limited coverage of AI concepts in the CSTA Computer Science Standards, the AI4K12 Initiative was launched in summer 2018 with the primary purpose of developing guidelines for what every student should know about and be able to do with AI. The development of these guidelines was modeled after the CSTA standards process (AAAI, 2018; Touretzky, 2018). While AI and data science were becoming common in media articles and higher education offerings at the time we started developing the AI4K12 guidelines, very little had been defined for K-12 students. However, a year after we began our work, the Introduction to Data Science Curriculum Framework (DSCF) was released in 2019 (IDSSP, 2019). The purpose of the Data Science curriculum framework is “to provide specifications for the development of a modern Data Science Curriculum that can be customized to local requirements” and be used for students in their last two years of high school. (IDSSP, 2019, p. 9). As such, the DSCF framework is organized into two units and its learning objectives broadly focus on helping stu- dents (1) understand and manipulate data, (2) develop a practical understanding of when data science can be used to solve problems, and (3) develop skills to use data science tools and techniques to solve problems. The developers of the DSCF state that while the framework gives students some preliminary skills in using a program- ming language and experience with automation, the main focus is on helping stu- dents understand the power of the tools and the importance of managing data care- fully (IDSSP, 2019). While the DSCF is meant as a guide to develop curriculum, its comprehensive articulation of learning objectives for Data Science knowledge also define what K-12 students should know and be able to do with data science in the current absence of standards or guidelines. There are several areas of overlap between the DSCF and the treatment of machine learning and societal impacts of AI in the AI4K12 Guidelines. Table 2 highlights the complementary and overlapping areas between the two frameworks. At the intersection of the DSCF and the AI4K12 Guidelines, there are significant overlaps in the areas of supervised and unsupervised learning such as: (1) selection of algorithms for classification or prediction; (2) partitioning of the dataset to evalu- ate the performance of the model; (3) creating decision tree classifiers and predic- tors; (4) training a classification or regression model; and (5) ethics of working with data and detecting and avoiding bias in datasets. The overlapping concepts described in the DSCF represent only a portion of the Big Idea 3 (Learning) concepts. The DSCF has a heavy focus on using ML to solve problems and using data science tech- niques to understand data and when to use different types of ML that goes beyond the AI4K12 Big Idea 3 guidelines. For example, the DSCF (1) explicitly asks stu- dents to pose classification and prediction questions, (2) provides explicit guidance on how to evaluate and measure performance of algorithms and models, and (3) provides opportunities for students to learn the steps in the data-handling pipeline and data wrangling techniques. Given the broader focus of the AI4K12 Guidelines on machine learning in general, the Big Idea 3 (Learning) guidelines help students understand how machine learning works through addressing three broad concepts: 13 Table 1 Hooks for Teaching AI within the 2017 CSTA Computer Science Standards 240 Concept Subconcept Relevant CSTA Standard Data & Analysis Collection, Visualization, & 2-DA-08 Collect data using computational tools and transform the data to make it more useful and reliable. (P6.3) 13 Transformation Inference & Models 1A-DA-07 Identify and describe patterns in data visualizations, such as charts or graphs, to make predictions. (P4.1) 1B-DA-07 Use data to highlight or propose cause-and-effect relationships, predict outcomes, or communicate an idea. (P7.1) 3A-DA-12 Create computational models that represent the relationships among different elements of data collected from a phenomenon or process. (P4.4) Algorithms & Programming Control 3A-AP-16 Design and iteratively develop computational artifacts for practical intent, personal expression, or to address a societal issue by using events to initiate instructions. (P5.2) Program Development Program Development 1B-AP-15 Test and debug (identify and fix errors) a program or algorithm to ensure it runs as intended. (P6.1, P6.2) 1B-AP-16 Take on varying roles, with teacher guidance, when collaborating with peers during the design, implementation, and review stages of program development. (P2.2) 2-AP-17 Systematically test and refine programs using a range of test cases. (P6.1) 3A-AP-21 Evaluate and refine computational artifacts to make them more usable and accessible. (P6.3) 3A-AP-23 Document design decisions using text,graphics, presentations, and/or demonstrations in the development of complex programs. (P7.2) Impacts of Computing Culture 1A-IC-16 Compare how people live and work before and after the implementation or adoption of new computing technology. (P7.0) 1B-IC-19 Brainstorm ways to improve the accessibility and usability of technology products for the diverse needs and wants of users. (P1.2) 2-IC-21 Discuss issues of bias and accessibility in the design of existing technologies. (P1.2) 3A-IC-25 Test and refine computational artifacts to reduce bias and equity deficits. (P1.2) Safety, Law, & Ethics 3A-IC-28 Explain the beneficial and harmful effects that intellectual property laws can have on innovation. (P7.3) 3A-IC-29 Explain the privacy concerns related to the collection and generation of data through automated processes that may not be evident to users. (P7.2) 3A-IC-30 Evaluate the social and economic implications of privacy in the context of safety, law, or ethics. (P7.3) International Journal of Artificial Intelligence in Education (2023) 33:233–266 Table 2 Intersection between the Curriculum Framework for Introductory Data Science (2019) & AI4K12 Guidelines (AI4K12.org) Areas of Alignment AI4K12 Guidelines Curriculum Framework for Introductory Data Science (DSCF) (1) Selection of algorithms for 3-A-iv: Nature of Learning: Topic Area 1.3 SUPERVISED LEARNING classification or prediction Constructing vs. using a reasoner Understand what sort of problems can be solved with clas- 3-A-vi.9–12 LO: Select the appropriate type of machine learning algo- sification rithm (supervised, unsupervised, or reinforcement learning) to solve a - Pose classification questions and identify situations that call reasoning problem for classification - Provide an algorithm to classify categorical outcomes - Pick an algorithm to classify categorical data - Pick an algorithm to predict based on 1–2 features (2) Partitioning of the dataset to 3-A-i.9–12 Unpacked: The cross-validation set is used to avoid Topic Area 1.3 SUPERVISED LEARNING evaluate the performance of overfitting. The test set consists of examples that were not used during Understand how algorithms/models are evaluated to measure the model training or for cross-validation, so it performance provides an unbiased prediction of the reasoner’s performance on new - Use software to calculate misclassification rates inputs - Compare classification algorithms/models and decide which 3-C-iii: Datasets: Bias is the better for a given situation based on total misclassifica- 3-C-iii.6–8 LO: Explain how the choice of training data shapes the tion rate behavior of the classifier, and how bias can be introduced if the train- - Explain how algorithms/models can be used to predict ing set is not properly balanced numerical outcomes, and explain how a goodness of fit 3-C-iii.9–12 LO: Investigate imbalances in training data in terms of measure can be used to quantify the success of the prediction gender, age, ethnicity, or other demographic variables that could result - Understand what overfitting is Understand how "set aside" data are used and why they are International Journal of Artificial Intelligence in Education (2023) 33:233–266 in a biased model, by using a data visualization tool important - Use a set-aside data set to compare classification algorithms/ models - Use a validation dataset to compare trees for prediction 241 13 Table 2 (continued) 242 Areas of Alignment AI4K12 Guidelines Curriculum Framework for Introductory Data Science (DSCF) 13 (3) Creating decision tree clas- 3-A-v: Nature of Learning: Adjusting internal representations Topic Area 1.3 SUPERVISED LEARNING sifiers and predictors 3-A-v.3–5 LO: Analyze a game where one constructs a decision tree, Understand what sort of problems can be solved with clas- describing the organization of the tree and the learning algorithm used sification to add nodes - Understand how Classification and Regression Trees (CART) 3-A-v.6–8 LO: Compare how a decision tree learning algorithm works are used to classify. Fit a tree, interpret and evaluate the vs. how a neural network learning algorithm performance works - Describe an algorithm to generate a tree to predict numerical outcomes using 1 or 2 features/variables - Fit and interpret a regression tree using software (4) Training a classification or 3-A-iii: Nature of Learning: Training a model Topic Area 1.3 SUPERVISED LEARNING regression model 3-A-ii.9–12 LO: Model how machine learning constructs a reasoner for - Understand how Classification and Regression Trees (CART) classification or prediction by adjusting the reasoner’s parameters (its are used to classify. Fit a tree, interpret and evaluate the internal representations) performance 3-A-iii: Nature of Learning: Training a model - Provide an algorithm to classify categorical outcomes 3-A-iii.K-2 LO: Demonstrate how to train a computer to recognize Topic Area 2.5 UNSUPERVISED LEARNING something - Explain the iterative process of one clustering algorithm such 3-A-iii.3–5 LO: Train a classification model using machine learning, as K-means and then examine the accuracy of the model on new inputs - Use software to prepare data, apply K-means clustering and 3-A-iii.6–8 LO: Train and evaluate a classification or prediction model interpret the results using machine learning on a tabular dataset 3-A-iii.9–12 LO: Use either a supervised or unsupervised learning algo- rithm to train a model on real world data, then evaluate the results International Journal of Artificial Intelligence in Education (2023) 33:233–266 Table 2 (continued) Areas of Alignment AI4K12 Guidelines Curriculum Framework for Introductory Data Science (DSCF) (5) Nature of data and datasets 3-A-iii: Nature of Learning: Finding patterns in data Topic Area 1.3 SUPERVISED LEARNING 3-A-ii.K-2 LO: Identify patterns in labeled data and determine the Express what data are and what they are used for Importance features that predict labels of data quality 3-A-ii.3–5 LO: Model how supervised learning identifies patterns in Topic Area 1.7 AVOID BEING MISLED BY DATA labeled data To provide students with a deeper understanding of how to 3-A-ii.6–8 LO: Model how unsupervised learning finds patterns in critique data and data-based claims, including an apprecia- unlabeled data tion of the ideas of bias, confounding and random error 3-C-i: Datasets: Feature sets Topic Area 2.5 UNSUPERVISED LEARNING 3-C-i.K-2 LO: Create a labeled dataset with explicit features to illustrate Distinguish between data that are appropriate for supervised how computers can learn to classify things like foods, movies, or toys versus unsupervised learning based on its structure, particu- 3-C-i.3–5 LO: Create a labeled dataset with explicit features of several larly the presence and roles of inputs and outputs types and use a machine learning tool to train a classifier on this data 3-C-i.6–8 LO: Create a dataset for training a decision tree classifier or predictor and explore the impact that different feature encodings have on the decision tree 3-C-i.9–12 LO: Compare two real world datasets in terms of the fea- tures they comprise and how those features are encoded 3-C-iii: Datasets: Bias 3-C-ii.3–5 LO: Illustrate how training a classifier for a broad concept such as "dog" requires a large amount of data to capture the diversity International Journal of Artificial Intelligence in Education (2023) 33:233–266 of the domain 3-C-ii.9–12 LO: Evaluate a dataset used to train a real AI system by considering the size of the dataset, the way that the data were acquired and labeled, the storage required, and the estimated time to produce the dataset 3-A-iv: Nature of Learning: Constructing vs. using a reasoner 3-A-iv.3–5 LO: Demonstrate how training data are labeled when using a machine learning tool 3-A-iv.9–12 LO: Illustrate what happens during each of the steps required when using machine learning to construct a classifier or predictor 243 13 Table 2 (continued) 244 Areas of Alignment AI4K12 Guidelines Curriculum Framework for Introductory Data Science (DSCF) 13 (6) Ethics of working with data 3-C-ii: Datasets: Large datasets Topic Area 1.3 SUPERVISED LEARNING and detecting and avoiding EU: A large dataset is typically required to capture the diversity of a - Heighten awareness of how ethical issues can arise in the bias in datasets complex domain and narrow down the range of possible reasoner various steps of the cycle (especially in the data gathering behaviors. There are multiple ways to construct, clean, and verify a stage) dataset. There can be large costs associated with creating the dataset - Heighten awareness of potential dangers of misuse and abuse and processing the data. Labeling training data is labor intensive and to pay specific attention to issues relating to data quality, may require specialized expertise (e.g., spotting disease in x-rays.) questioning skills, and presentation skills Bias can be introduced during each step of dataset creation Topic Area 1.1 SOCIAL & ETHICAL ISSUES 3-C-iii: Nature of Learning: Training a model - Express what data are and where they come from 3-C-iii.K-2 LO: Examine a labeled dataset and identify problems in the - Identify and describe errors in decisions and predictions data that could lead a computer to make incorrect predictions owing to faulty use of data 3-C-iii.3–5 LO: Examine features and labels of training data to detect - Discuss how and when data can support making decisions potential sources of bias - Provide examples of the social and personal consequences of 3-C-iii.6–8 LO: Explain how the choice of training data shapes the predictions derived from models built on data behavior of the classifier, and how bias can be introduced if the train- ing set is not properly balanced 3-C-iii.9–12 LO: Investigate imbalances in training data in terms of gender, age, ethnicity, or other demographic variables that could result in a biased model, by using a data visualization tool Big Idea 5 Societal Impacts International Journal of Artificial Intelligence in Education (2023) 33:233–266 International Journal of Artificial Intelligence in Education (2023) 33:233–266 245 Nature of Learning, Neural Networks, and Datasets. The AI4K12 guidelines extend student learning beyond the learning objectives covered in the data science cur- riculum framework into understanding (1) differences between human learning and machine learning; (2) how neural networks are constructed and work; and (3) the difference between constructing versus using a reasoner. Other Relevant Educational Standards Current K-12 standards such as the Common Core (CC) and the Next Generation Science Standards (NGSS) offer rigorous guidance for teaching STEM concepts, and developmentally appropriate introductions to inquiry, analysis, and problem- solving, that can help prepare students for learning AI. But they provide no AI con- tent knowledge. The NRC Framework for K-12 Science Education, which informed the development of the NGSS, identifies five Science and Engineering Practices that align well with the skills students need to engage with AI (NRC, 2012). They are (1) Asking questions (for science) and defining problems (for engineering); (2) Devel- oping and using models; (3) Planning and carrying out investigations; (4) Analyzing and interpreting data; and (5) Constructing explanations and designing solutions. For example, in the 3–5 grade band students are expected to be able to “develop and/ or use models to describe and/or predict phenomena” (National Research Council, 2013a. p. 387), which aligns well with the AI4K12 Guideline 3-A-iii which reads “Train a classification model using machine learning, and then examine the accu- racy of the model on new inputs” (AI4K12.org, 2020). In addition, we identified several cross-cutting themes within NGSS that align well with the skills students need to engage deeply with AI concepts and understand how AI systems work. For example, NGSS’s “Patterns” theme provides a foundation in skills for observing “patterns of forms and events that guide organization and classification.” Observa- tions of patterns “prompt questions about relationships and the factors that influ- ence them” (National Research Council, 2013b. p. 413). This cross-cutting theme aligns with AI4K12 Guideline 3-A-ii which reads “Identify patterns in labeled data and determine the features that predict labels” (AI4K12.org, 2020). Table 3 provides additional examples of AI-enabling practices within the NGSS’s Science & Engi- neering Practices, NRC’s Cross-cutting themes, and Common Core’s English and Language Arts (ELA) and Literacy standards. One of the goals of the AI4K12 Guidelines and the Five Big Ideas in AI is to equip students to understand, evaluate, and explain the societal impacts of AI-ena- bled technologies (Touretzky et al., 2019b). Within the Common Core Standards (CCS) there are several English and Language Arts and Literacy (ELA/Literacy) standards that align well with this goal, such as “RI.3.3 Describe the relationship between a series of historical events, scientific ideas or concepts, or steps in techni- cal procedures in a text, using language that pertains to time, sequence, and cause/ effect (3-LS3-1)” (NGA Center for Best Practices & CCSSO, 2010a). We want stu- dents to understand the technical steps in developing AI-enabled systems and the places where human decision-making can have immediate and long-term effects on the people who use those systems. For example, students should be able to 13 Table 3 Select NGSS, NRC Framework for K-12 Science, and Common Core Standards that are relevant to AI instruction 246 Framework Practice or Cross-Cutting Theme Examples of Standards NGSS 1. Patterns Observed patterns of forms and events guide organization and classification, 13 (NSTA, 2014) and they prompt questions about relationships and the factors that influence them Similarities and differences in patterns can be used to sort, classify, and analyze simple rates of change for natural phenomena 4. Systems and System Models Defining the system under study—specifying its boundaries and making explicit a model of that system—provides tools for understanding and test- ing ideas that are applicable throughout science and engineering In 3–5, A system can be described in terms of its components and their interactions In 6–8, Models can be used to represent systems and their interactions—such as inputs, processes and outputs—and energy, matter, and information flows within systems Science and Engineering Practices: NRC Framework Asking Questions (for Science) and Defining Problems (for In K-2, students can ask questions based on observations to find more infor- for K-12 Science Education (NRC, 2012) Engineering) mation about the natural and/or designed world(s) In 3–5, students can ask questions that can be investigated and predict reason- able outcomes based on patterns such as cause and effect relationships Developing and Using Models Modeling in 3–5 builds on K–2 experiences and progresses to building and revising simple models and using models to represent events and design solutions Develop and/or use models to describe and/or predict phenomena Planning and Carrying out Investigations In 6–8, students can develop and/or revise a model to show the relation- ships among variables, including those that are not observable but predict observable phenomena In 9–12, students can develop and/or use a model (including mathemati- cal and computational) to generate data to support explanations, predict phenomena, analyze systems, and/ or solve problems Analyzing and Interpreting Data Analyzing data in 3–5 builds on K–2 experiences and progresses to introducing quantitative approaches to collecting data and conducting multiple trials of qualitative observations. When possible and feasible, digital tools should be used Analyze and interpret data to make sense of phenomena using logical reasoning Constructing Explanations and Designing Solutions Generate and compare multiple solutions to a problem based on how well International Journal of Artificial Intelligence in Education (2023) 33:233–266 they meet the criteria and constraints of the design solution Table 3 (continued) Framework Practice or Cross-Cutting Theme Examples of Standards Common Core ELA/Literacy RI.3.3Describe the relationship between a series of historical events, sci- entific ideas or concepts, or steps in technical procedures in a text, using language that pertains to time, sequence, and cause/effect. (3-LS3-1) W.3.2Write informative/explanatory texts to examine a topic and convey ideas and information clearly. (3-LS3-1) SL.3.4 “Report on a topic or text, tell a story, or recount an experience with appropriate facts and relevant, descriptive details, speaking clearly at an understandable pace. (3-LS3-1) International Journal of Artificial Intelligence in Education (2023) 33:233–266 247 13 248 International Journal of Artificial Intelligence in Education (2023) 33:233–266 identify how the choice of data used to train a decision-making system will impact the system’s predictions and affect the lives of the populations who use that system (Touretzky et al., 2019b). The AP Computer Science Principles Exam is one of the first computer science exams to require students to explain design choices, computing technologies, and algo- rithms at the K-12 level (College Board, 2020). Many of the AI4K12 Guidelines also ask students to explain concepts, such as 3-A-iv Constructing vs. Using a Reasoner, which reads “Explain the difference between training and using a reasoning model.” Guidelines like these align well with the CCS “W.3.2 Write informative/explanatory texts to examine a topic and convey ideas and information clearly (3-LS3-1)” (NGA Center for Best Practices & CCSSO, 2010b). Moreover, we want students to be able to read critically about AI, both in news reports and in case-studies of AI technolo- gies (Touretzky et al., 2019b). This skillset builds on CCS SL.3.4 “Report on a topic or text, tell a story, or recount an experience with appropriate facts and relevant descriptive details, speaking clearly at an understandable pace (3-LS3-1)” (NGA Center for Best Practices & CCSSO, 2010c). While ELA/Literacy skills are not common in computer science and technical courses, these skills are essential for students to be able to criti- cally explore AI and understand the limits and the boundaries of their own knowledge. Five Big Ideas in AI With little external guidance from the literature on the content and scope of AI edu- cation for K-12 students, the AI4K12 Steering Committee (David Touretzky, Chris- tina Gardner-McCune, Fred Martin, and Deborah Seehorn) began their work in 2018 by formulating a list of “Five Big Ideas in AI” that would serve as the organ- izing framework for the guidelines (Touretzky et al., 2019a). This was modeled after the CSTA Computing Standards, which are organized around five major concepts as described earlier (CSTA, 2017). Figure 2 is an infographic illustrating the Five Big Ideas in AI. Summarized briefly, the ideas are: 1. Perception: Computers perceive the world using sensors. Perception is the extrac- tion of meaning from sensory signals, using knowledge. 2. Representation and Reasoning: Agents maintain representations of the world and use them for reasoning. Representation drives reasoning; reasoners operate on representations. 3. Learning: Computers can learn from data. A machine learning algorithm con- structs a reasoner by adjusting the internal representations of a reasoning model, such as a decision tree or a neural network. voice 4. Natural Interaction: Intelligent agents require many kinds of knowledge in order recognition. to interact naturally with humans. This includes knowledge about language, “common sense” and cultural knowledge, and knowledge about human emotions. 5. Societal Impact: AI can impact society in both positive and negative ways. Rele- vant issues include fairness, bias, and transparency of automated decision-making systems, economic impacts of automation, cultural aspects of living with intel- ligent machines, and applications of AI for social good 13 International Journal of Artificial Intelligence in Education (2023) 33:233–266 249 An informational poster designed for classroom use that describes the Five Big Ideas was released in 2019 and has been translated into 16 languages. These are available at the AI4K12.org website. Guidelines Development Process The Steering Committee assembled a Working Group composed of 16 K-12 teach- ers, plus 9 individuals with other types of expertise such as AI research, CS educa- tion research, curriculum development, and district and state leadership. The K-12 teachers were organized in four grade bands (K-2, 3–5, 6–8, and 9–12). Each had multiple years of experience teaching children in their grade band; most were still working in the classroom (n = 13), but a few had moved on to teacher training or administrative positions (n = 3). However, most of the teachers had little or no famil- iarity with AI when they joined the project. Five people stepped down at the end of year 1, and we recruited three replacements. In addition, an Advisory Board was formed that eventually grew to a dozen members representing industry, academia, the US government, nonprofits, and overseas curriculum development projects (AI4K12.org, 2022). Figure 3 outlines the guidelines development process. The working group addressed the five big ideas one at a time. For each big idea, we held tutorial ses- sions to familiarize members with the relevant AI concepts. These were supple- mented by select videos, unplugged activities, and online demonstrations to deepen their engagement with the concepts. Following these large group sessions, each grade band held its own weekly or bi-weekly video meetings, and its members also worked asynchronously, to develop concept treatments for the focal big idea that set out the major concepts and subconcepts and accompanying skills. A key contribu- tion of the K-12 working group members at this stage was their evaluation of the developmental appropriateness of various AI concepts through alignment with con- tent, skills, and practices used in teaching general STEM and CS topics. Following this grade band work, we held monthly video meetings of the entire Working Group where the grade bands discussed their formulations of the AI concepts and skills appropriate for their students. Initially these were presented as slide decks; later they were assembled into spreadsheets. Through discussions of the grade band work we identified key themes and potential misconceptions in the concept treatments, and checked alignment with STEM and computing concepts and socio-emotional learn- ing goals. After the grade bands incorporated this feedback, the next step was to merge the four grade band concept treatments into a unified spreadsheet, aligning topics and language as much as possible. The result gave a blueprint for the level of depth at which the concepts should be addressed. The grade bands were given time to do cross-grade band work to smooth out the horizontal progression, and within their own grade band spent some time smoothing out the vertical progression. By the end of Year 2, the steering committee and advisory board recognized that our understanding of the concepts students should be learning had evolved over time and there were some concepts that were underdeveloped, others 13 250 International Journal of Artificial Intelligence in Education (2023) 33:233–266 Fig. 2 The “Five Big Ideas in AI” infographic developed by the AI4K12 Initiative overdeveloped, and some that were missing. In addition, there were several places where lower grade bands were proposing more advanced work than the succeed- ing grade band, while in other places adjacent grade bands were doing the same thing. There were also editorial concerns about use of consistent language across the grade bands. Therefore, the Steering Committee drafted a revised synthesis document by starting with a fresh concept list and importing ideas from the uni- fied spreadsheet produced by the grade bands, aligning rows and columns to use uniform language. The synthesis document was then critiqued by members of the Working Group and after revisions, it was released for public comment. Further revisions are anticipated in response to comments received. As of this writing, the Big Idea 1 (Perception) progression chart is undergoing revision, and Big Ideas 2 (Representation and Reasoning), 3 (Learning), and 4 (Natural Interaction) are undergoing public review. The synthesis for Big Idea 5 (Societal Impact) is in development. 13 International Journal of Artificial Intelligence in Education (2023) 33:233–266 251 Fig. 3 The guidelines development process The grade band progression chart for each big idea is a table in which the rows are concepts and subconcepts and the columns are the four grade bands. For each cell in the table, the guideline offers a Learning Objective (what students should be able to do) and an Enduring Understanding (what students should know). These are sometimes accompanied by Unpacked descriptions that amplify the LO or EU. Figure 4 shows the progression for the “Humans vs. machines” subconcept of the “Nature of Learning” main concept in the Big Idea 3 progression chart. Big Idea 3: What Every K‑12 Student Should Know About Machine Learning Big Idea 3 (Learning) is intended to immerse students in the richness and com- plexity of Machine Learning. The first step is to demystify what machine learning is and how it works. To understand this students need to acquire four essential insights about machine learning: 1. Definition of machine learning: Machine learning allows a computer to acquire behaviors without people explicitly programming those behaviors. This para- phrases a widely-cited definition often attributed to Arthur Samuel (Wikipedia, 2021, fn. 2), who coined the term “machine learning” (Samuel, 1959). Using learning to induce a behavior is very different from the usual approach to con- structing computing applications by explicitly programming each step. 2. How machine learning algorithms work: Learning new behaviors results from changes the learning algorithm makes to the internal representations of a rea- soning model, such as a decision tree or a neural network. The essential insight here is that the learning algorithm is constructing a reasoner. In a decision tree, this means adding new nodes. In a neural network it means adjusting the weights. What we want students to understand is that this kind of learning is a simple, mechanical process; there is no self-awareness or any kind of magic involved. More ambitious types of learning, such as learning new concepts or learning 13 252 International Journal of Artificial Intelligence in Education (2023) 33:233–266 Fig. 4 The progression for Big Idea 3, Concept A: Nature of Learning, subconcept i: Humans vs. machines by demonstration, while the subject of current research, are not included in the guidelines because they are not yet successful enough to have real-world impact 3. The role of training data: When the reasoning model is capable of a great variety of behaviors, large amounts of training data are required to narrow down the learn- ing algorithm’s choices. Popular accounts of machine learning often emphasize the massive amounts of data required, but this is not always true. Common misconcep- tions hold that high dimensional input spaces or large numbers of output classes require large training sets. It is actually model complexity that drives the need for data. If the desired reasoning behavior can be described by a simple model, a small amount of data may suffice. But classifiers with complex decision boundaries and function approximators with highly nonlinear input–output relationships cannot be realized by simple models. What we want students to understand is that people are able to learn from small numbers of examples because they use their prior knowl- edge and intuition to choose models that favor “sensible” solutions. Computers are not yet good at this. Therefore the computer must resort to complex models with many parameters. And accurately estimating these parameter values requires many training examples. Structured architectures such as convolutional neural networks or transformer networks are attempts to incorporate prior knowledge into models in ways that are sensible but not too restrictive. 4. Learning phase vs. application phase: The reasoner constructed by the machine learning algorithm can be applied to new data to solve problems or make deci- sions. In the learning phase, the learning algorithm is constructing a reasoner using the training data. When it runs the reasoner on the training data and assesses the results, the focus is on improving the reasoner’s behavior. In the application phase the reasoner is no longer tied to the learning algorithm or training data. It is simply processing inputs and producing outputs. 13 International Journal of Artificial Intelligence in Education (2023) 33:233–266 253 The Big Idea 3 progression chart (AI4K12.org, 2020) expands these essential insights by introducing students to three major concepts. Table 4 lists these three concepts and the subconcepts contained in each. Section A: Nature of Learning This first section introduces the major concepts in machine learning and the skills students need to develop. The latter include training a reasoning model and measur- ing the model’s accuracy on new inputs. 3-A-i: Humans vs. machines. It is important for students to appreciate that machines do not learn the way humans do. Both people and computers can learn by finding patterns in data, but humans are flexible learners who also learn in other ways, such as by direct instruction, by observing others, by asking ques- tions, by experimentation, or by making connections to past experience. In con- trast, computers use specialized algorithms that learn to solve narrowly defined problems. In the K-8 grades we ask students to make this distinction. In 9–12 we ask them to define the three major categories of machine learning algorithms (supervised, unsupervised, and reinforcement learning) and draw analogies with human learning situations, e.g., supervised learning is like having a coach telling you what you’re doing wrong. 3-A-ii: Finding patterns in data. This is the essence of machine learning. In K-2 and 3–5 we ask students to identify patterns in labeled data by inspection. In K-2 they express the rules informally, while in 3–5 they draw a decision tree. In 6–8 we introduce unsupervised learning and ask students to model how clustering works. In 9–12 we delve deeper into supervised learning and ask students to model how classification or prediction problems are solved by an incremental learning algorithm, by manually adjusting the parameters of a polynomial regression model to reduce the overall error. This is an analogy to the iterative gradient descent optimization used in the backpropagation learn- ing algorithm, but without having to actually measure the error gradient. 3-A-iii: Training a model. Every student should have the experience of training a reasoning model using a machine learning tool. In K-2 this would be a highly scaffolded exercise using a tool like Teachable Machine (Phillips, 2019) to clas- sify gestures or sounds. In higher grades students might use tabular data and a tool like Machine Learning For Kids (Lane, 2021a). In 9–12 we emphasize work- ing with real-world datasets, so students will be asked to train a predictor, classi- fier, or clusterer using a publicly available real-word dataset. 3-A-iv: Constructing vs. using a reasoner. This row is concerned with two topics: the distinction between the learning phase and the application phase in machine learning (the fourth essential insight mentioned earlier), and, in grades 9–12, all the steps involved in constructing a reasoner and assessing the accuracy of the trained reasoning model, including use of cross-validation and test datasets. 3-A-v: Adjusting internal representations. This row explores the second essen- tial insight, that learning algorithms work by adjusting the internal representa- 13 254 International Journal of Artificial Intelligence in Education (2023) 33:233–266 Table 4 Big Idea 3 (Learning) concepts (A) Nature of Learning (B) Neural Networks i. Humans vs. machines i. Structure of a neural network ii. Finding patterns in data ii. Weight adjustment iii. Training a model (C) Datasets iv. Constructing vs. using a reasoner i. Feature sets v. Adjusting internal representations ii. Large datasets vi. Learning from experience iii. Bias tions of a reasoning model. In grades 3–5 students reflect on how decision tree learning works by successively adding nodes at the fringe of the tree. In 6–8 they compare decision tree learning with neural network learning algorithms, while in 9–12 they also consider parameter adjustment in regression algorithms and policy adjustment in reinforcement learning. 3-A-vi: Learning from experience. This row covers reinforcement learning. In 3–5 we want students to know about some of the applications of reinforcement learning, e.g. that computers can become expert game players by playing against themselves. In 6–8 students are asked to explain the difference between super- vised learning and reinforcement learning, while in 9–12 they are asked to select the most appropriate learning algorithm (supervised, unsupervised, or reinforce- ment learning) for various types of problems. Students should recognize that reinforcement learning is appropriate for sequential decision problems. Section B: Neural Networks As a type of reasoning model neural networks are properly included in Big Idea 2 (Rep- resentation and Reasoning). But due to the prominent role deep neural networks play in modern machine learning, and the fact that neural network reasoners are almost exclu- sively constructed by learning algorithms, we decided to cover them in Big Idea 3. 3-B-i: Structure of a neural network. This row covers the neural network as a reason- ing model. Its components include connections, weights, and units with nonlinear transfer functions. In grades 3–5 students are introduced to simple networks of one to three binary threshold units. In grades 6–8 they learn about input, hidden, and out- put layers, and come to see neural networks as “wiggly” functions that map inputs to outputs in complex ways. In grades 9–12 they learn about more sophisticated neural network architectures such as recurrent and convolutional networks, and Generative Adversarial Networks (GANs) that produce deepfakes. 3-B-ii: Weight adjustment. In this row students learn how changing the weights of a neural network alters its behavior. In grades 3–5 they make the weight adjustments themselves to get a network of 1–3 units to implement a desired behavior. In grades 6–8 they learn a simple weight adjustment rule that applies to single-layer networks. In grades 9–12 they use a backpropagation learning tool such as TensorFlow Play- ground (Sato, 2016) to train a multilayer network and examine the changes that occur to the weights and hidden unit response functions. However, we do not expect stu- 13 International Journal of Artificial Intelligence in Education (2023) 33:233–266 255 dents to learn the mathematics underlying backpropagation, as this involves vector calculus. What we expect them to understand is that there is an equation for calculat- ing an error signal, and that this signal originates at the output layer, is propagated backward to earlier layers, and is used to adjust the weights coming into each layer. Machine learning electives for advanced students in grades 11–12 might explore backpropagation in greater depth and even derive the weight update equation via the chain rule, but for most students this level of detail is too much. Section C: Datasets In the past decade, progress in machine learning has produced dramatic advances in artificial intelligence applications including speech recognition, computer vision, and machine translation. These advances have resulted from the ability to train large models on massive datasets. But in some cases, training on problematic datasets has produced systems that exhibit bias against certain classes of persons. Thus, it is important to under- stand how the choice of dataset affects the outcome of a machine learning experiment. 3-C-i: Feature sets. This row introduces the idea of describing training instances as collections of features and explores how the choice of feature encoding can influence learning. In grades 9–12 students may examine real-world datasets (e.g., demographic or financial data) and compare the encoding schemes used. 3-C-ii: Large datasets. This row examines why large datasets are needed for com- plex problems, and investigates techniques and costs of labeling large amounts of training data. Students may explore historically and scientifically significant data- sets that are browsable online, such as the ImageNet and Coco datasets for object recognition. In grades 9–12 they may evaluate large public datasets to estimate their size and the cost of assembling, cleaning, and labeling the data. 3-C-iii: Bias. This row explores the effects of bias in a training set, a topic which is further explored in Big Idea 5 (Societal Impact). One common source of bias results from use of a non-representative sample, e.g., training a face detector on a dataset skewed heavily toward Caucasian males may not produce acceptable performance when the detector is tested on other types of people. Another poten- tial source of bias results from the learning algorithm picking up on correlations that reduce the classification error rate for the training set but result in disparate treatment of certain groups based on historical factors that, for reasons of fairness, should not be considered in decision making. In grades 9–12 students are exposed to data exploration techniques that can uncover imbalances in a training set. Experiences We Want Students to Have Training a model We tell students that machines learn by finding patterns in data. One way they can experience this firsthand is by training a classifier to discriminate between classes they already understand. For example, a student could use Google’s 13 256 International Journal of Artificial Intelligence in Education (2023) 33:233–266 Teachable Machine (Phillips, 2019) to construct a visual classifier to distinguish between a peace sign, a thumbs up gesture, and a “no gesture” baseline condition. Good recognition rates can be achieved with training images that take only a cou- ple of minutes to collect. This is a compelling demonstration of a computer mak- ing complex discriminations that feel intuitive to the student. The computer seems “smart”. A crucial follow-up experiment is to test the robustness of the classifier by seeing how well it does if the student makes the gestures with their other hand, or with their hand held upside-down, or while wearing different clothing. The com- puter may seem less smart after exposing its limited generalization abilities. This is an example of how experimentation can lead students to a deeper understanding of complex phenomena. Learning an unfamiliar concept from labeled examples Visual classifiers built from deep neural networks, like Teachable Machine, are black boxes: their internal repre- sentations are not observable. There is another kind of machine learning experience we want students to have, where the representations are visible. We want them to understand what it “feels like” to discover patterns in data, i.e. to be the learner, learning to perform a classification task by exposure to labeled examples. For this we must choose classes the student is not already familiar with, so that their learning will be driven purely by the training data, not prior knowledge. This can be done as early as K-2 by asking students to learn a discrimination between cartoon drawings of fish, some of which are labeled “poisonous” and some labeled “safe”. The fish have a variety of attributes such as body color, head color, head shape, fin shape, etc. Categories can be simple (e.g., purple fish are poisonous), but for older students they can be made arbitrarily complex (e.g., purple fish with square heads, or red fish with either round heads or yellow tails, are poisonous). As the space of admissible hypotheses grows, more training examples are required to determine the correct one. In K-2 students are only required to verbalize the rule they come up with, but in later grades we may ask them to express their reasoning more formally, such as by con- structing a decision tree, which we discuss next. Constructing a decision tree using labeled data We can revisit the experience of concept learning in more depth in grades 3–5 by having students construct actual decision trees to perform the classification. This exposes them to the idea of formal- izing knowledge as a data structure, and the tree is an example of an internal repre- sentation used by a reasoner. At this stage we’re still using cartoon images where the feature set is implicit. At the next stage, grades 6–8, we introduce students to the notion of designing an explicit feature set for describing training instances. For example, they might invent a set of features for describing toys, or movies. At this stage they can be introduced to an automated decision tree learning tool such as MachineLearningForKids (Lane, 2021a), and can examine the decision trees that it constructs. Since they are able to create their own training sets, they can experiment to see how changes in the training data are reflected in changes to the decision tree. 13 International Journal of Artificial Intelligence in Education (2023) 33:233–266 257 Manual parameter optimization for regression learning (simulating how a neural net learns by weight adjustment) Decision tree learning is a good place to start because symbolic data structures are more intuitive, but a lot of the heavy lifting in machine learning today is done by neural networks trained by gradient descent learning algorithms. So another kind of experience we want students to have is what it “feels like” to optimize a nonlinear function by parameter tuning. We can do this without introducing gradients, and we would like students to adjust multi- ple parameters without having to deal with multidimensional inputs. So rather than have them adjust the weights of a neural net, we envision them solving a regression problem such as visually fitting a cubic polynomial to a plot of noisy data by tweak- ing sliders controlling its coefficients. Students could be asked to visually judge the quality of the fit, or they could be given some assistance in the form of a con- tinuous display of sum squared error. This exercise would help students intuitively appreciate the complexity of searching a high dimensional parameter space and of working with nonlinear functions, and the strategy of searching the space by mak- ing small changes. The hope is that they would develop a mental model of gradient descent optimization that they can apply to their understanding of neural networks without having to delve into the complexities of the backpropagation learning rule. Exploring historically important datasets Certain datasets have played a prominent role in the development of machine learning. Inviting students to explore these data- sets online helps them connect with the history of the field, and gives them a feel for the complexities of the training sets used to solve real-world problems. Examples include the MNIST dataset of handwritten digits (LeCun et al., 2021), the ImageNet dataset used in many image classification competitions (Deng et al., 2009), and the CoCo object detection, segmentation, and captioning dataset (Lin et al., 2014). Both ImageNet and Coco have web interfaces that facilitate online browsing. Training on real‑world datasets One of the themes we emphasize in the 9–12 grade band is the use of real-world datasets to show students how the con- cepts they’re studying apply to problems in everyday life. Thus, when students in grades 9–12 are asked to train a decision tree classifier or a neural network regression model, we suggest that they do so in the context of solving a problem of practical importance using real data. This could be financial, medical, socio- economic, or consumer data. The growing interest in high school data science courses, and in broadening K-12 mathematical education to include more statis- tics, aligns with this goal. Online machine learning repositories such as Kaggle offer public domain datasets suitable for these exercises. Tools For Exploring Machine Learning in K‑12 Several types of software tools support the learning experiences we want students to have. First, we distinguish between “black box” demonstrations that allow students to train a classifier, predictor, or clusterer but provide no insight into its operation, 13 258 International Journal of Artificial Intelligence in Education (2023) 33:233–266 and “glass box” demonstrations that expose the reasoning model’s internal represen- tations and allow students to observe how they change with training. The quintessential black box machine learning demonstration is Teachable Machine (Phillips, 2019). It allows untrained users to quickly train a visual classifier using webcam images or audio input. Internally it combines a pre-trained deep neu- ral network architecture called MobileNet with some trainable weights at the output layer, but users have no way of knowing this. Another popular tool for training vari- ous types of classifiers is MachineLearningForKids (Lane, 2021a), which is based on the IBM Watson AI service. Although mainly a black box demo, in cases where decision tree learning is used, MachineLearningForKids does offer a way to graphi- cally display the decision tree. Code.org’s AI Lab is another easy to use machine learning tool that does not require programming (Code.org, 2021). It uses a k-near- est neighbors algorithm rather than decision trees, and it can perform either classifi- cation or prediction. AI Lab includes a collection of built-in datasets but also allows users to upload their own data from a CSV file. What students get from black box learning demonstrations is, first of all, the expe- rience of deciding on a set of classes and assembling a training set of labeled exam- ples. And second, a chance to test the success of their training by measuring the trained model’s performance on test inputs. In addition, some of these tools include an option to export the trained classifier as a module that can be incorporated into a JavaScript, Python, Scratch, or MIT App Inventor program, thus allowing stu- dents to build AI-powered artifacts of their own. See Gresse von Wangenheim et al. (2021) for a review of 16 interactive machine learning tools for K-12. A prime example of a glass box demonstration is Google’s TensorFlow Play- ground, which provides a visualization of a multi-layer neural network classifier undergoing training via the backpropagation learning algorithm. To facilitate visu- alization, the input domain is restricted to points in the (x,y) plane, and there are just two classes, represented by a single output unit. The user can control the number of hidden layers and the number of units in each layer. Both the weights between units and the response functions of the hidden units are graphically represented and change as learning progresses. The tool includes built-in training sets for four classi- fication problems of increasing complexity, from simple linearly-separable clusters, to circle/surround, to XOR, to nested spirals. Glass box demonstrations give insight into how the algorithm works. In the case of TensorFlow Playground, students can see hidden units develop their response func- tions as training progresses, and they will observe that units in later layers develop more complex decision boundaries by building on the units in earlier layers. The drawback to glass box demonstrations is that they are typically limited to toy prob- lems that are amenable to visualization. But some tools in this category may provide a window into more complex models, e.g., an online implementation of a convolu- tional neural network for digit recognition might display all the feature maps. Hybrid approaches are also possible. Hitron et al. (2019) present evidence that uncovering just selected aspects of a black box machine learning exercise, giving students a rough mental model of the process, results in enhanced understanding. 13 International Journal of Artificial Intelligence in Education (2023) 33:233–266 259 Relationship to the Other Big Ideas in AI Big Idea 2, Representation and Reasoning, sets the stage for Big Idea 3 by introduc- ing the notions of classes of reasoning problems (classification, prediction, etc.) and families of reasoning algorithms for solving those problems. Big idea 2 also encom- passes the relationship between reasoning and representation, which mirrors that of algorithms and data structures in computer science: representations drive reasoning, and reasoning algorithms operate on representations. Machine learning then intro- duces a new class of algorithm, the learning algorithm, that operates on a reasoner’s internal representations. There is also substantial contact between Big Idea 3 and Big Idea 5, Societal Impact, due to concerns about the use of machine learning tools to construct auto- mated decision making systems that may negatively impact people’s lives, e.g., sys- tems that score credit or employment applications or predict criminal recidivism rates. Problems with these systems can result from the use of biased or unrepresent- ative training data, or from the use of historically accurate training data that reflects the results of past societal biases against certain marginalized groups. In Big Idea 1, Perception, we define “perception” as the extraction of mean- ing from sensory signals using knowledge. Machine learning is now the dominant approach for constructing these extraction mechanisms. Speech understanding and visual object recognition are two examples of perceptual tasks that have become sig- nificantly more reliable in the last decade as a result of advances in neural network learning. Big Idea 4, Natural Interaction, covers a broad range of topics including natural language understanding, emotion recognition, and commonsense reasoning. As with perception, recent progress in machine learning has led to advances in some of these areas. For example, many state of the art natural language applications, including machine translation and question answering systems, are built using neural networks trained on huge corpuses. Desired Implementation of the AI4K12 Guidelines into Classrooms Full implementation of the guidelines in the U.S. will require changes to state edu- cation standards to amend the requirements for universal K-12 computing educa- tion that are gradually being adopted today. Throughout the elementary grades (K-5) the guidelines are designed to be integrated into existing course work, as there are few opportunities to take a stand-alone elective in these grades. We envision AI being taught in K-2 through unplugged activities, teacher-guided demonstrations, and interactive experiences with AI technologies such as Alexa. This also holds for grades 3–5, but as students begin programming in block-based frameworks such as Scratch, they can take advantage of AI extensions to these frameworks to incorpo- rate capabilities such as speech or object recognition into their programs, allowing them to discover simple truths about AI. 13 260 International Journal of Artificial Intelligence in Education (2023) 33:233–266 Starting in middle school (grades 6–8) we imagine that students might have the opportunity to take a stand-alone AI elective course or for AI to be integrated into existing programming courses. In grades 9–12, students can take stand-alone AI and computing courses and/or enroll in AI or computing Career Technical Education (CTE) pathways commonly offered at US-based high schools. CTE pathways are designed to help students explore their career interests and prepare for either col- lege or employment. They consist of 3–4 courses that must be completed alongside industry certifications to verify students’ knowledge. CTE courses often attract a broad audience because they equip students with employable skills. Statewide CTE pathways in AI have been implemented in both Florida (Florida Department of Edu- cation, 2022) and Georgia (Georgia Department of Education, 2021). An AI exam for CTE students is available from Certiport (2021) as part of their IT Specialist Certification program. AI is also being incorporated into more general computing courses. Exploring Computer Science (ECS), an introductory high school computing curriculum designed to attract a broad range of students to computing, offers an optional AI module that teachers can integrate into their course (Clark, 2019). Likewise, Carnegie Mellon’s Python-based CS Academy includes an optional AI module in the CS2 course that introduces students to classification, prediction, and neural networks (CMU CS Acad- emy, 2021). We also imagine that AI can be integrated into existing AP Computer Sci- ence Principles and AP CS A courses, though this has not yet occurred. Discussion of AI should not be confined to computing courses. AI topics intersect with many parts of the curriculum, including mathematics, language arts, science, social studies, music, and art. Introducing AI in these areas will require first educat- ing non-CS teachers about these topics. For example, art teachers will be excited to learn how machine learning-powered tools allow humans to partner with machines to explore new avenues for creative expression. Examples include neural style trans- fer (Dumoulin et al., 2017), line drawing to image translators (Isola et al., 2017), and text-to-image transformers (Saharia et al., 2022). Conclusion In this paper we have described the process of developing the AI4K12 Guidelines and painted an ambitious picture of what AI in K-12 could look like. We have also presented a novel view of how machine learning should be taught to K-12 students. First, our approach emphasizes the distinction between a reasoning model and a learning algorithm. The job of the learning algorithm is to construct a reasoner. It does this by adjusting the parameters of a reasoning model based on training data. This framing of the learning process as parameter adjustment cov- ers a wide range of reasoning models, including decision trees, neural networks, k-means clustering, and Q-table driven action selection. Second, when considering the kinds of learning experiences students should have, we include not just training a reasoner, but also being the learning algo- rithm: trying to find patterns in data, or trying to adjust model parameters to 13 International Journal of Artificial Intelligence in Education (2023) 33:233–266 261 minimize output error. We believe this will help students develop better mental models of how learning algorithms work. Third, while machine learning was once considered an advanced topic in com- puter science, the grade band progression chart developed by the AI4K12 Work- ing Group shows that even K-2 students can begin learning the rudiments of this subject, such as recognizing patterns in a data set. If AI literacy is the new com- puter literacy, kindergarten is not too early to start. The AI4K12 guidelines propose a content progression — one that has not yet been tested in classrooms. Future research in AI education will need to develop more detailed learning progressions to describe how students approach these concepts, how their understanding develops over time, and the types of miscon- ceptions they are prone to. Acknowledgements We are grateful to our colleague Fred Martin, now an emeritus member of the AI4K12 Steering Committee, for his important contributions to launching the AI4K12 Initiative and for- mulating the Five Big Ideas. We also thank the members of the AI4K12 Working Group and Advisory Board for their diligent efforts in developing the guidelines. Funding This work was supported by the National Science Foundation under Grant No. DRL-1846073. Declarations Competing interests The authors have no relevant financial or non-financial interests to disclose. References AAAI. (2018). AAAI Launches “AI for K-12” Initiative in collaboration with the Computer Science Teachers Association (CSTA) and AI4All. https://aaai.org/Pressroom/Releases/release-18-0515.pdf. Accessed 18 Oct 2022 AI4K12.org. (2020). Big Idea 3 - Learning - K-12 Learning Progression. Release for Public Review November 19, 2020. https://ai4k12.org/wp-content/uploads/2021/01/AI4K12-Big-Idea-3-Progr ession-Chart-Working-Draft-of-Big-Idea-3-v.11.19.2020.pdf. AI4K12.org. (2022). Working Group and Advisory Board Members. https://ai4k12.org/working-group- and-advisory-board-members/. Accessed 18 Oct 2022 Ali, S., DiPaola, D., Lee, I., Sindato, V., Kim, G., Blumofe, R., & Breazeal, C. (2021). Children as creators, thinkers and citi

Machine Learning and the Five Big Ideas in AI_removed.pdf

Document Details

Related

Full Transcript

Upgrade to continue