Lecture 3: A Brief History of AI PDF

Lecture 3 A Brief History of AI: Precursors, Gestation and Birth of AI Rob Gaizauskas COM1005 2023-24 Lecture Outline • Scene Setting – Reviewing the Definition of AI – Principal Sub Areas of AI – Disciplines Contributing to AI • Historical Overview Precursors (… – 1943) Gestation and Birth (1...

Lecture 3 A Brief History of AI: Precursors, Gestation and Birth of AI Rob Gaizauskas COM1005 2023-24 Lecture Outline • Scene Setting – Reviewing the Definition of AI – Principal Sub Areas of AI – Disciplines Contributing to AI • Historical Overview Precursors (… – 1943) Gestation and Birth (1943 – 1956) Golden Early Years (1956-1969) The First “AI Winter” (1966-73) Rise of Knowledge-based and Expert Systems (1969-1989) New Paradigms: Connectionism; Intelligent Agents; Embodied AI (1986 – present) – Scientific Method, Big Data and Deep Learning (1987 – present) Reading: * = mandatory – *Russell and Norvig (2021), Chapter 1 “Introduction” – *Wikipedia: History of Artificial Intelligence. https://en.wikipedia.org/wiki/History_of_artificial_intelligence – – – – – – • COM1005 2023-24 Scene Setting Reviewing the Definition of AI • Last week we considered several definitions of AI: – McCarthy: (AI) is the science and engineering of making intelligent machines, especially intelligent computer programs. It is related to the similar task of using computers to understand human intelligence, but AI does not have to confine itself to methods that are biologically observable. – Whitby: Artificial Intelligence (AI) is the study of intelligent behaviour (in humans, animals and machines) and the attempt to find ways in which such behaviour could be engineered in any type of artifact. COM1005 2023-24 Scene Setting Reviewing the Definition of AI • These definitions presuppose some notion of “intelligence” • While not agreeing entirely on what constitutes intelligence, psychologists concur that it includes the abilities to – – – – – – Reason Learn Plan Understand complex ideas Solve Problems Adapt to the environment COM1005 2023-24 Scene Setting Principal Sub Areas of AI • These abilities correspond remarkably well with what have emerged as the principal sub areas of AI – – – – – – – Deduction, reasoning, problem solving Knowledge representation Planning Learning Natural language processing Motion and manipulation Perception COM1005 2023-24 Scene Setting Disciplines Contributing to AI • AI is not simply a creation of computer scientists • AI has drawn on ideas, techniques and perspectives of wide range of disciplines including: – – – – – – – – Philosophy Mathematics Economics Neuroscience Psychology Computer Engineering Control Theory & Cybernetics Linguistics • See Russell &Norvig, Chapter 1.2 COM1005 2023-24 Lecture Outline • Scene Setting – Reviewing the Definition of AI – Principal Sub Areas of AI • Historical Overview Precursors (… – 1943) Gestation and Birth (1943 – 1956) Golden Early Years (1956-1969) The First “AI Winter” (1966-73) Rise of Knowledge-based and Expert Systems (1969-1989) New Paradigms: Connectionism; Intelligent Agents; Embodied AI (1986 – present) – Scientific Method, Big Data and Deep Learning (1987 – present) – – – – – – COM1005 2023-24 Lecture Outline • Scene Setting – Reviewing the Definition of AI – Principal Sub Areas of AI • Historical Overview This week Next week Precursors (… – 1943) Gestation and Birth (1943 – 1956) Golden Early Years (1956-1969) The First “AI Winter” (1966-73) Rise of Knowledge-based and Expert Systems (1969-1989) New Paradigms: Connectionism; Intelligent Agents; Embodied AI (1986 – present) – Scientific Method, Big Data and Deep Learning (1987 – present) – – – – – – COM1005 2023-24 Why Look at History of AI? • History of ideas primarily, not of events, dates and people – Many key concepts in modern AI better understood by knowing context in which they were first introduced. • • • • Neural nets Search space Combinatorial explosion Expert system • • • • Machine/deep learning Theorem proving Big Data Embodiment/Intelligent Agents • Evolution of ideas in AI not linear – ideas once abandoned (e.g. neural nets) may suddenly become relevant again • ``Those who cannot remember the past, are condemned to repeat it.’’ George Santayana, The Life of Reason: Five Volumes in One COM1005 2023-24 Precursors (... – 1943) • We saw last week that stories/models of artificial beings manufactured or brought to life to function as assistants of various forms have pervaded human literature and preoccupied engineers for millenia and across many very different cultures • By 1943 a number of developments had set the stage of the emergence of AI as a scientific discipline: – The appearance of the first electronic computers – Turing’s foundational work relating deduction and computation, building on early 20th work on symbolic logic – Advances in understanding of the functions and activities of neurons in the brain COM1005 2023-24 Gestation and Birth (1943 – 1956) • McCulloch-Pitts Neuron (1943) • Turing’s “Computing Machinery and Intelligence” paper (1950) • Logic Theorist (1955-56) • Dartmouth Conference (1956) COM1005 2023-24 Gestation and Birth (1943 – 1956) McCulloch-Pitts Neuron • 1943 paper “A Logical Calculus of Ideas Immanent in Nervous Activity” generally recognised as the 1st work in AI • Drew on – Knowledge of basic neurophysiology, esp. that the brain is a network of neurons that fire in allor-nothing pulses based on inputs – Formal propositional logic (Russell and Whitehead) – Turing’s theory of computation COM1005 2023-24 Gestation and Birth (1943 – 1956) McCulloch-Pitts Neuron (cont) • Neurons (or nerve cells) are electrically excitable cells that process and transmit information through electrical and chemical signals http://en.wikipedia.org/wiki/Neuron COM1005 2023-24 Gestation and Birth (1943 – 1956) McCulloch-Pitts Neuron (cont) • Proposed a model of an artificial neuron Dendrite Soma Axon http://en.wikipedia.org/wiki/Artificial_neuron COM1005 2023-24 Gestation and Birth (1943 – 1956) McCulloch-Pitts Neuron (cont) • Proposed a model of an artificial neuron Dendrite Soma Axon http://en.wikipedia.org/wiki/Artificial_neuron – Inputs (x1,…,xn), each with associated weight (wk1,…,wkn) are fed into neuron νk n – Neuron computes weighted sum of inputs ∑ wi xi and passes it through a i=0 non-linear transfer function φ to give output yk – Typically φ is a step-like function which produces a 1 as output if the weighted sum of inputs exceeds a threshold defined by wk0 and 0 COM1005 2023-24 otherwise Gestation and Birth (1943 – 1956) McCulloch-Pitts Neuron (cont) • McCulloch and Pitts showed – all logical connectives (and, or, not, etc.) – indeed, any computable function could be computed by a network of connected artificial neurons • Donald Hebb (1949) proposed an updating rule for modifying the connection weights in artificial neurons allowing a network to be trained – Hebbian learning still an influential model today COM1005 2023-24 Gestation and Birth (1943 – 1956) McCulloch-Pitts Neuron (cont) • In 1951 two Harvard undergrads Marvin Minsky and Dean Edmonds built the 1st neural network computer – the SNARC – 3000 vacuum tubes; simulated 40 neurons – Used it to model the behaviour of a rat in a maze searching for food One Neuron from the Snarc http://cyberneticzoo.com/?p=1053 • Neural network research remains a core part of AI today – recent explosion of interest “deep learning” (= neural nets + GPUs) – Minsky continued to play a central role in this sub-field from 1950 till his death in 2016 COM1005 2023-24 Gestation and Birth (1943 – 1956) Turing’s “Computing Machinery and Intelligence” paper (1950) • Last week saw how Turing proposed to replace the question “can machines think?” with a question framed in terms of the imitation game • Paper was immensely influential: not only introduced an operational test for thinking but introduced other ideas that continue to influence AI – Machine learning – Genetic algorithms – Reinforcement learning COM1005 2023-24 Gestation and Birth (1943 – 1956) Logic Theorist (1955-56) • Written by Herb Simon, Alan Newell and J.C. Shaw • “first program deliberately engineered to mimic the problem solving skills of a human being” (http://en.wikipedia.org/wiki/Logic_Theorist) • Generally called “the first AI program” • Proved 38 of the first 52 theorems in Whitehead and Russell's Principia Mathematica and found a shorter/more elegant proof for one theorem than previously known • Later said: [We] invented a computer program capable of thinking non-numerically, and thereby solved the venerable mind-body problem, explaining how a system composed of matter can have the properties of mind COM1005 2023-24 Gestation and Birth (1943 – 1956) Logic Theorist (1955-56) • Given – Some axioms (self-evident truths) of formal logic, A – Any theorems previously proved, P – A candidate theorem to prove, T LT worked by applying one 4 methods: – Substitution – tries to change one logic expression (e.g. T) into another, logically equivalent one (e.g. an axiom in A), by substitution of variables or replacements of connectives – Detachment – to prove T create sub-goals to prove expressions S → T and S (NB: read S → T as “If S then T”) – Chaining forward – if T has the form A → C then create sub-goals to first prove expression A → B and then prove B → C – Chaining backward -- if T has the form A → C then create sub-goals to first prove expression B → C and then prove A → B COM1005 2023-24 Gestation and Birth (1943 – 1956) Logic Theorist (1955-56) • Introduced several key concepts to AI: – Reasoning as search: proof is viewed as a search starting from a hypothesis root node, expanding it along different branches according to deductive rules and stopping when the proposition to be proved is obtained. – Heuristics: proof tree grows exponentially and hence some branches need to be pruned using “rules of thumb” or “heuristics” to constrain the search space while hopefully not losing the path to a solution. – List processing: to implement the Logic Theorist the authors developed a list processing programming language, IPL, that served as the basis for the Lisp programming language introduced slightly later by John McCarthy – an important AI programming language still used today. COM1005 2023-24 Reasoning as Search: Proof Search Tree Root node New state obtained from old state by application of deductive rules … Each node = proof state at that point • At root, state = {hypotheses} • At subsequent nodes, state = {hypotheses + derived propositions} … … … Proposition to be proved COM1005 2023-24 Gestation and Birth (1943 – 1956) Dartmouth Conference (1956) • Proposed by John McCarthy, Marvin Minsky, Nathaniel Rochester and Claude Shannon: “We propose that a 2 month, 10 man study of artificial intelligence be carried out during the summer of 1956 at Dartmouth College in Hanover, New Hampshire. The study is to proceed on the basis of the conjecture that every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it. An attempt will be made to find how to make machines use language, form abstractions and concepts, solve kinds of problems now reserved for humans, and improve themselves. We think that a significant advance can be made in one or more of these problems if a carefully selected group of scientists work on it together for a summer.” • Also attended by Ray Solomonoff, Oliver Selfridge, Trenchard More, Arthur Samuel, Herbert A. Simon, and Allen Newell – All went on to be AI leaders over the following decades. • Generally taken as the birth point of AI COM1005 2023-24 Summary • AI draws on many fields beyond Computer Science • AI as a field is generally held to have started at the Dartmouth Conference in 1956 • But various previous developments made the time ripe for that very conference: – The creation the first general purpose programmable computers (late 1940s) – McCulloch and Pitts work on the artificial neuron (1943) – Turing’s paper “Computing Machinery and Intelligence” (1950) – Simon and Newell’s Logic Theorist (1955-56) COM1005 2023-24