Introduction to Artificial Intelligence PDF - Maastricht University

Introduction to Artificial Intelligence Week 1, Part 1: Course logistics and general intro Dr. Aki Härmä Department of Advanced Computing Sciences, Faculty of Science and Engineering, Maastricht University Teachers, course logistics, materials Teachers Dr. Aki Härmä Dr. Guangzh...

Introduction to Artificial Intelligence Week 1, Part 1: Course logistics and general intro Dr. Aki Härmä Department of Advanced Computing Sciences, Faculty of Science and Engineering, Maastricht University Teachers, course logistics, materials Teachers Dr. Aki Härmä Dr. Guangzhi Tang Dr. Dennis Tim Dick Assistant professor Assistant professor Soemers Lecturer Course coordinator From China Assistant professor From Germany From Finland Educated in US From Maastricht Data modalities Was 19 years at Edge AI Game AI, Search, and Multimodal Philips Research Reinforcement processing ML, Neural Learning NLP, audio, ML Network, Deep Learning Search & Introduction Optimisation closing Teaching assistants Dumitru Giacomo Taisia Martin Čajka, Mikołaj Verşebeniuc, Anerdi, Pimenova, From Slovakia Czaplejewicz, From Moldova From Italy From Saint- From Poland Bachelor Thesis Bachelor of Interested in Petersburg, on Quantized - CNNs, ML, Data Science NLP and CV Russia, Neural Econometrics and Artificial Recommender Networks Intelligence and AI Master's student Systems and Master of Deep Learning Artificial in general Intelligence in Maastricht Books (recommended but not required) Poole & Mackworth, Artificial Intelligence: Foundations of Computational Agents, 3rd ed, 2023 Russell & Norvig: Artificial Intelligence: A modern approach, 4th ed, 2021 Or use ChatGPT/Gemini  Week 1 Introduction and overview (Aki) Course schedule What is AI, history, and what technical fields are “inside” The EU AI act and other frameworks and related concepts & logistics Python intro (those who need), First tutorial session Week 2 Machine learning (Aki and Guangzhi) Basic definitions and paradigms Classic algorithms and performance Monday Main lecture hall MSM (16-18) Tutorial session, building and testing a machine learning system 2 x 45 minutes lecture on week’s theme Week 3 Deep learning (Guangzhi) Introduction of the weekly quizzes From conventional machine learning to deep learning We’ll check if online participation is possible Basic architectures Tutorial: design and train an autograd system Friday Submission deadlines (23.59CET) Week 4 Search and optimization (Dennis) Weekly quiz closed Search strategies, optimization targets Min-max, path finding Submission of the Tutorial processing results/code Beam search Week 5 Modalities (Tim) Tuesday Tutorials (9-12) Text, image, speech, knowledge graphs Translations between modalities Work on the weekly assignment supported by Tas Data fusion Two cohorts in rooms C0.008/C0.0016 Week 6 Foundation models and responsible AI (Aki) Answers/discussion about last week’s quizzes Prompting large models to have tasks done Responsible AI Artificial General Intelligence Grading Participation in the Tutorials 30% - minimum 50% (repair possible) - Online participation possible Answering correctly to the Canvas quizzes 30% - minimum 50% Exam (40%) The use of generative AI technology for quizzes and tutorial reports is allowed (and even recommended) Course communications Discord channel: https://discord.gg/evAYGzBUtx Github (tbd) for shared code The slides of the day’s lecture will be in Canvas, at the latest, a day before the lecture (not necessarily final versions) Intelligence and agency Intelligence Latin Intellegere ~ capacity to understand or comprehend “Intelligence measures an agent’s ability to achieve goals in a wide range of environments.“ (S. Legg and M. Hutter, 2003, based 70 different definitions of intelligence in the literature) “general intelligence g” Spearman’s two-factor model (~1904) g “specific abilities s” - e.g., 56 different s language math visual social Artificial Intelligence (AI textbooks) “Artificial Intelligence is a study of computational agents that act intelligently.” (Poole & Mackworth, 2023) “In this book, we adopt the view that intelligence is concerned mainly with rational action. Ideally, an intelligent agent takes the best possible action in a situation. We study the problem of building agents that are intelligent in this sense.” (Russell & Norvig, 2021) “Artificial intelligence is the simulation of human intelligence processes by machines, especially computer systems, enabling them to perform tasks such as learning, reasoning, problem-solving, and decision-making.” (GPT-4o, 2024) Intelligence and artificial intelligence “Intelligence measures an agent’s ability to achieve goals in a wide range of environments.“ (Legg & Hutter) “Artificial Intelligence is a study of computational agents that act intelligently.” (Poole & Mackworth) Here's the black and white illustration depicting the humanoid robot on the deserted island. The robot is utilizing natural resources to construct makeshift solar panels, with wind turbines in the background and a radio transmitter sending distress signals. (GPT-4o) Artificial Intelligence (EU definitions) “Artificial intelligence (AI) systems are software (and possibly also hardware) systems designed by humans that, given a complex goal, act in the physical or digital dimension by perceiving their environment through data acquisition, interpreting the collected structured or unstructured data, reasoning on the knowledge, or processing the information, derived from this data and deciding the best action(s) to take to achieve the given goal. AI systems can either use symbolic rules or learn a numeric model, and they can also adapt their behaviour by analysing how the environment is affected by their previous actions.” “Simply put, AI is a collection of technologies that combine data, algorithms and computing power.” eur-lex.europa.eu/legal- content/EN/TXT/PDF/?uri=CELEX:52020DC0065 Artificial Intelligence (EU definitions) “‘AI system’ means a machine-based system that is designed to operate with varying levels of autonomy and that may exhibit adaptiveness after deployment, and that, for explicit or implicit objectives, infers, from the input it receives, how to generate outputs such as predictions, content, recommendations, or decisions that can influence physical or virtual environments;” Later: “AI systems that distinguish it from simpler traditional software systems or programming approaches and should not cover systems that are based on the rules defined solely by natural persons to automatically execute operations” Article 3: Definitions | EU Artificial Intelligence Act Are birds AI systems? A pigeon can learn to click the right button to get food - Search for the right button - Learn the right button (infer, predict) A pigeon can also perform simple computations AI systems are machines (but not necessarily human-made) The EU AI Act Foundations for the regulation of AI in the EU Defines acceptable / non- acceptable AI systems Defines governance, post- market monitoring Penalties and fines for companies violating the regulations EU AI Act Compliance Checker | EU Artificial Intelligence Act Artificial Intelligence (OECD) The framework classifies AI systems and applications: People & Planet, Economic Context, Data & Input, AI Model and Task & Output …. AI Model: This is a computational representation of all or part of the external environment of an AI system – encompassing, for example, processes, objects, ideas, people and/or interactions that take place in that environment. Core characteristics include technical type, how the model is built (using expert knowledge, machine learning or both) and how the model is used (for what objectives and using what performance measures). OECD Framework for the classification of AI systems (oecd-ilibrary.org) The Dutch university perspective Intelligence is often defined as the ability to reason with knowledge, to plan and to coordinate, to solve problems, to perceive, to learn and to understand language and ideas. … The term Artificial Intelligence as used in this document refers to the study of intelligence, whether artificial or natural, by computational means * van der Meulen, A., Kwisthout, J., ten Teije, A., Schlobach, S., van Splunter, S., Winands, M., van Netten, S., Visser, A., van Someren, M., Dastani, M., & Dignum, F. (2018). Frame of Reference - Bachelor’s and Master’s Programmes in Artificial Intelligence: The Dutch Perspective. Kunstmatige Intelligentie Opleidingen Nederland (KION). Computational Intelligence “Computational Intelligence (CI) is the theory, design, application and development of biologically and linguistically motivated computational paradigms. Traditionally the three main pillars of CI have been Neural Networks, Fuzzy Systems and Evolutionary Computation. ” What is Computational Intelligence? - IEEE Computational Intelligence Society Intelligence Augmentation (IA) Assisting technologies to support human cognitive capabilities - Computers we use in general - Augmented reality interfaces - Brain-implants Augmented reality Neuralink Synthetic Expertise | SpringerLink Swarm Intelligence (SI) Swarm intelligence (SI) is the collective behavior of decentralized, self- organized systems, natural or artificial. Human culture, ant colony, etc. History of AI * Good Old-Fashioned Artificial Eugenics, cognitive Intelligence (GOFAI) science, linguistics, logic Artificial Intelligence* 18th – 19th “Artificial Intelligence” century Machine learning, Mathematics, computational optimization, statistics signal processing and control, Control and signal processing, generative models pattern recognition, computing 1950’s 2000’s 2020’s History of (GOF)AI Noam Chomsky (1928-), Modern linguistics, semantics Artificial Intelligence “AI winter” 1974- Nature & Nurture Dartmouth workshop (1956), Francis Galton (-1911), John McCarthy et al. Father of eugenics and intelligence testing “… every aspect of learning or any other feature of (human) intelligence can in principle be so precisely described that a machine can be made Rudolph Carnap (-1970), logical positivism to simulate it.”  Artificial (GOF)AI or symbolic AI Intelligence “… every aspect of learning or any other feature of (human) Symbolic Logic Description intelligence can in principle be so Semantic precisely described that a machine Logic web can be made to simulate it.” Rule-based expert Heuristic systems Knowledge search Rule-based systems for describing graphs the human decision making, Logic and search methods to Still important for eXplainable AI, XAI implement them … because rule-based decisions are easy to understand for a human Cybernetics, computation, machine learning Cybernetics - "the study of systems of Statistical Euler, any nature which are capable of learning theory Legendre, receiving, storing, and processing Gauss, information so as to use it for control“* Modern neural Newton,.. networks Alan Turing (-1954) Claude Shannon (-2001) Automaton theory Turing machines *Andrey Theory of communication Kolmogorov (-1987) Norbert Wiener, (-1964) Geffrey Hinton Yann LeCun Control and signal processing, pattern recognition, computing Artificial Intelligence (a critical view) “AI is an ideological invention that covers various technologies of advanced computing sometimes in an incoherent manner” (Stefan Popenici, 2022) “In an interesting reversal, it is Wiener’s intellectual agenda that has come to dominate in the current era, under the banner of McCarthy’s terminology.” (Michael Jordan, 2022) Norbert Wiener’s “cybernetics” - a general vision for intelligent systems based on sensing/control/optimization (1948) John McCarthy coined the term “artificial intelligence” (1956) as a property of a machine that imitates human intelligence Artificial Intelligence (embodied, expressed) Noncurated Machine learning Unfair detection and decisions train predict biased data system (gender, race, …) Is this “Artificial Unfairness” or real unfairness “embodied” in a machine? Is “intelligence” an embodied Intelligent property of a machine or an X behavior y expression interpreted by a human? The Turing imitation game The system A passes the test if the human C cannot determine whether A or B is a human On the terminology of this course “ARTIFICIAL INTELLIGENCE” Logics Computational Intelligence Symbolic AI Formal Knowledge Fuzzy Logic reasoning Representations Genetic and evolutionary algorithms Decision trees Neural networks Control theory Reinforcement learning Foundation models Machine learning Signal processing On the terminology of this course Signal processing: Agent: - Filtering, modeling, preprocessing of - A system that has a behavior sensor data, images, audio Behavior: Machine learning: - The reactions of a system to internal - Learning from data, understanding and external stimuli - Computational modeling of data Knowledge: Optimization: - Structured data based on a world- - Fit data to a model (incl. search) model - Find the best model World model: Information retrieval: - Model of the environment of an agent - Search / discover from data The large models Are GPTs, Gemini’s, LLAMA’s the end of GOFAI, Logics, Signal Processing, Machine learning,..?  The most complex machine humans have ever built More than trillion, that is, 1000000000000, moving parts!* * A common estimate of the number of parameters in GPT-4 Trained in a self- supervised way 15 trillion tokens* – 150 million books of text! A book pile the size of Paul-Henri Spaaklaan 1 C-tower! Why we still need to learn “old technologies” The LLM technology is a proof of a concept: - shows that we can build systems that understand human language - can perform similar reasoning task as human, draw pictures, write text But - The technical solution of LLMs/LMMs is scientifically ugly, economically badly scalable, and an environmental catastrophe. Conventional intelligent algorithms - Scale well, are explainable for humans, can be easily minimized - For a task, equivalent conventional solutions are orders of magnitude more efficient! Symbolic AI and logics Agency, Knowledge representation, propositional and first-order logic Rational agent PEAS model - P – performance measure - E – environment - A – Actuators - S – Sensors Rational agent Goal-based agent Takes an action based on a rule Rule is based on some logic system Propositional Logic (Boolean reasoning) Proposition 𝑃𝑃, 𝑄𝑄 , e.g., “Tom eats fruits”, “Manchineel is a fruit” Negation ¬𝑃𝑃 says that 𝑃𝑃 is not true: ⊤, it is false: ⊥ Conjunction 𝑃𝑃 ∧ 𝑄𝑄 says that both are true Disjunction 𝑃𝑃 ∨ 𝑄𝑄 states that at least one of the two is true Implication 𝑃𝑃 ⇒ 𝑄𝑄 says that 𝑄𝑄 follows from 𝑃𝑃 Biconditional 𝑃𝑃 ⇔ 𝑄𝑄, implication both ways Does it hold that 𝑃𝑃 ∧ 𝑄𝑄 ⇒ 𝑅𝑅 where 𝑅𝑅 is “Tom eats Manchineels”? Manchineel fruit (incl. deadly toxins) Propositional Logic (Boolean reasoning) Propositions 𝑃𝑃 = “Tom eats all edible fruits” (⊤) 𝑄𝑄 = “Manchineel is an edible fruit”(⊥) With 𝑃𝑃 ∧ 𝑄𝑄 ⇒ 𝑅𝑅 Is 𝑅𝑅 = “Tom eats Manchineels” a True proposition? Proposition are atomic (indivisible) statements that are either True or False Manchineel fruit (incl. deadly toxins) Propositional logic Atomic Propositions that are either ⊤or ⊥: P =“Tom eats fruits” Propositional connectives: Connective Read as Java/Python Name ¬ Not ! Negation ∧ And & Conjunction ∨ Or | Disjunction ⇒ Implies if(q): p Implication ⇔ If and only if if(q): unique p Biconditional ⊤ True True Truth ⊥ False False Falsity First-order logic (or predicate logic)  Objects or terms (Tom, fruit, apple, manchineel, edibility, sickness)  Predicates that describe properties of objects HasProperty(Apple, edible)  Functions that map objects to one another Eating(Tom, BadFruit) ⇒ sickness  Quantifiers that allow us to reason about multiple objects ∀ fruit, or ∃ fruit  The same logical connectives as in propositional logic Examples of Inferences: ∃𝑥𝑥. (Is(x, Fruit) ∧ Eating(Tom, x)⇒sickness) Fruit is is ∃𝑥𝑥. (Is(x, Fruit) ∧ HasProperty(x, edible)) Apple Manc. contains ∀x. Eating(Tom, ¬contains(x, Phorbol)) ⇒ happy hasProperty likes Phorb. Edible eats Tom  First-order logic A knowledge graph A world model consisting of triplets Fruit is is Apple Manc. Term (object) Relation Term (object) contains hasProperty Apple Is Fruit likes Edible Phorb. Manchineel Is Fruit eats Tom Apple hasProperty Edible Tom eats Edible conditionally Manchineel contains Phorbol Happy Sick Tom likes Fruit Description logic Individual case, Grounded statement (Bbox) Ontology (Tbox) “Tom eats an apple”⇒ happy Thing Thing is is is Dead is Dead Phorb. Phorb. contains contains Fruit Animal Apple Animal eats is eats is hasProperty hasProperty Edible Human Edible Tom Happy Sick Happy Sick Description logic Individual case, Grounded statement (Bbox, “instance”) Ontology (Tbox, “Class”) “Tom eats an manchineel”⇒ sick, dead Thing Thing is is is Dead is Dead Phorb. Phorb. contains contains Fruit Animal Manc. Animal eats is eats is hasProperty hasProperty Edible Human Edible Tom Happy Sick Happy Sick Description logic 2001 Applications today: Rule engines and graph queries Business rule engines Reservation systems Legal/medical protocols Graph query frameworks Fuzzy and probabilistic logics, and causal inference Computational Intelligence and statistics Fuzzy logic “Crisp Logic” - Boolean logic - everything is either True or False “Fuzzy Logic” – Truth value is a continuous variable in [0,1] Even an apple is not always edible “A fuzzy membership Edibility function” Maturity Fuzzy inference system Fuzzy Set Theory (L. A. Zadeh, 1969) Mamdani Fuzzy Inference System Takagi-Sugeno Fuzzy Model (TS Method) Applications today: Fuzzy logic Control systems in consumer electronics Control system of a washing machine A Blood pressure meter from 1994 In 2021 Probabilistic logics Reasoning under uncertainty Kolmogorov’s probability axioms. First: Apple 𝑃𝑃 𝑒𝑒 ≥ 0 𝑃𝑃(𝑒𝑒𝐴𝐴 ) 𝑃𝑃(𝑒𝑒𝐵𝐵 ) for any event e in an event space 𝐸𝐸. Second: Edible Inedible Ε 𝑃𝑃 𝑒𝑒 = 1 𝑒𝑒 Third: 𝑃𝑃 𝑒𝑒𝐴𝐴 ∨ 𝑒𝑒𝐵𝐵 ∨ ⋯ = 𝑃𝑃 𝑒𝑒𝐴𝐴 + 𝑃𝑃 𝑒𝑒𝐵𝐵 + ⋯ for mutually exclusive events 𝑒𝑒𝐴𝐴 , 𝑒𝑒𝐵𝐵 , …! Conditional probability and Bayes’ rule 𝑃𝑃(𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸 ∧ 𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹) All plants 𝑃𝑃 𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸 𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹 = 𝑃𝑃(𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹) 𝑃𝑃 𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹 𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸 = 𝑃𝑃(𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸 ∧ 𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹) All Fruits 𝑃𝑃(𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸) Bayes’ rule: Edible 𝑃𝑃 𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹 𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸 𝑃𝑃(𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸) 𝑃𝑃 𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸 𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹 = 𝑃𝑃(𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹)  Bayes' Network 𝑃𝑃 𝐴𝐴 = 0.3 𝑃𝑃 𝑀𝑀 = 0.05 Apple Manch. Directed Acyclic Graph (DAG) Edges are conditional Apple Manch. 𝑷𝑷(𝒆𝒆𝒆𝒆𝒆𝒆𝒆𝒆) true true 0.3 probabilities true false 0.01 Tom eats a manchineel false true 0.4 Examples false false 0.01 𝑃𝑃 𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠 𝑇𝑇𝑇𝑇𝑇𝑇 𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒 𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚, 𝑔𝑔𝑔𝑔𝑔𝑔𝑔𝑔𝑔𝑔 𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎 = 0.6 ∗ 0.01 ∗ 0.3 = 0.018 𝑃𝑃 ℎ𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎 𝑑𝑑𝑑𝑑 𝑛𝑛𝑛𝑛𝑛𝑛 𝑒𝑒𝑒𝑒𝑒𝑒, 𝑔𝑔𝑔𝑔𝑔𝑔𝑔𝑔𝑔𝑔 𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚 Sick Happy = 0.7 ∗ (1 − 0.4) ∗ 0.05 = 0.021 Eats 𝑷𝑷(𝒔𝒔𝒔𝒔𝒔𝒔𝒔𝒔) Eats 𝑷𝑷(𝒉𝒉𝒉𝒉𝒉𝒉𝒉𝒉𝒉𝒉) 𝑃𝑃 ℎ𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎 𝑑𝑑𝑑𝑑 𝑛𝑛𝑛𝑛𝑛𝑛 𝑒𝑒𝑒𝑒𝑒𝑒, 𝑛𝑛𝑛𝑛𝑛𝑛 𝑔𝑔𝑔𝑔𝑔𝑔𝑔𝑔𝑔𝑔 𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚 = 0.7 ∗ (1 − 0.4) ∗ (1 − 0.05) = 0.4 true 0.6 true 0.1 false 0.05 false 0.7  Causal inference Simulated Randomized Controlled Apple Manch. Trials (RCT) Counterfactual reasoning: what-if Tom do Tom eats a logic; do-notation not eat m. manchineel 𝑃𝑃(𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠|𝐝𝐝𝐝𝐝 𝑒𝑒𝑒𝑒𝑒𝑒 𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚 ) Sick Happy Able to prove based on data, for example, giving Tom a fruit does not cause him to get sick A break!