Cognitive Science: Representation and Computation

Cognitive Science

• An interdisciplinary field encompassing cognitive psychology, neuroscience, and artificial intelligence
• Also includes philosophy, linguistics, anthropology, sociology, and economics
• Focuses on internal representations and computation
• Uses the information processing metaphor, viewing the mind as an information processor

Friedenberg's Approach

• Discusses representation and computation directly
• Does not discuss the roots behind these concepts

Representation

• Includes symbolic, digital vs. analog (dual-code), and propositional forms

Computation

• Involves computations on representations
• Explains all information processing

Marr's Tri-Level Hypothesis (1982)

• Computational Level: Requires a clear specification of the problem and emphasizes the purpose or reason for the adaptive or learned process.
• Algorithmic Level: Centers on how to execute the information process, focusing on form (symbolic) over meaning and the manipulation of representations.
• Implementation Level: Concerns the hardware level and what the information processor is made of.

Cognitive Psychology

• Rooted in cognitive psychology
• Cognitive psychology is a cognitive approach, which is different from behaviorism and psychodynamic approaches.
• Cognition involves mental activity, acquisition, storage, transformation, and use of knowledge.
• Cognition occupies a major portion of human psychology.
• Cognitive psychology helps understanding of other areas of psychology and disciplines outside psychology.
• It provides an "owner's manual" for your mind.
• Cognitive psychology helps understand data analysis and supports humans in data-driven decision-making.

Organization of Cognition

• The book discusses different perspective approaches, including philosophical, psychological, cognitive, neuroscience, and linguistic perspectives.
• Concepts like representation are started directly.
• The study approach starts from basic processes (perception/attention) via memory to representation and reasoning.
• Slides are leading for the exam and the chapter provides background information.

Cognitive Psychology Defined

• Cognitive psychology studies cognizing by individual humans, including:
  • perception (Ch 4)
  • attention (Ch 4)
  • memory (Ch 5)
  • knowledge (Ch 5)
  • language (Ch 9)
  • reasoning (Ch 5, AI part)
  • problem solving (Ch 5, AI part)
  • decision-making (Ch 5/8)

Philosophical Antecedents: The Greeks

• Plato (427-347 BC)
  • Two 'worlds': observable, imperfect world and the eternal, changeless, abstract world of perfect forms/ideas
  • One world is physical and ending where body resides and the Mind/soul is part of the abstract eternal world (dualism).
  • Knowledge is through rationalism where trough good thinking through mathematics and philosophy, knowledge can be gained.
  • Mind knows everything (nativism), just remember
• Aristotle (384-322 BC)
  • Guided by observations of natural and biological processes.
  • Rejected Plato’s dualism
  • The solution is one world, the observable reality
  • Change is central where we learn and change by observation (empiricism).

Plato versus Aristotle (Comparison)

• Mind/Body:
  • Plato believed in Dualism
  • Aristotle believed in Monism
• Knowledge:
  • Plato emphasized Rationalism
  • Aristotle emphasized Empiricism.
• Origin of Mind:
  • Plato proposed Nature (Nativism).
  • Aristotle proposed Nurture.

Dualism, Free-Will, and Consciousness

• Chapter 2 of Friedenberg gives historical context with these debates
  • Debate 1: Turing’s view on intelligence by a computer
  • Debate 2: Searle’s Chinese room argument

Friedenberg: Psychological Approach (Chapter 3)

• Empirical or deductive method is used
• Uses dependent, independent variables and controlled experiments

Voluntarism / Structuralism

• Wilhelm Wundt (1832 -1920)
  • Father of scientific psychology
  • Established 1st Psychology Lab in 1875 in Germany
  • Decomposed the mind into structural/mental elements and study these separately
  • Physiological Psychology (experimental) relation between sensation and perception through introspection (immediate experience in the book)
  • Higher order processes and social processes cannot be studied by experimentation because this is too complex and introspection is too fallible (mediate experience in the book)

Early Memory Research

• Zeitgeist (Wundt): Memory is a higher order process and shouldn't be studied experimentally
• Ebbinghaus studied his own memory, learning nonsense syllables (ZAF BUQ QUJ)

Functionalism

• William James (1842 –1910) wrote "Principles of Psychology"
• Introspection has issues where one has to freeze the mind
• He argued that Psychologist Fallacy meant the same experience is thought to be same in different minds
• The mind is a stream of consciousness
• Aim is to understand functional relationships between stimuli and responses
• Focuses on everyday experiences rather than decompose cognition in structural aspects

Behaviorism (Mostly US)

• Watson (1878-1958)
  • Inspired by Functionalism
  • Studied the association between stimulus and responses
  • Rejected introspection and studied observable behavior only
  • Darwin’s influence: Animal experiments
• Skinner (1904-1990)

John B. Watson

• Founder of Behaviorism
• Total rejection of mentalism in scientific psychology (e.g. thinking is simply silent speech).

B. F. Skinner’s Radical Behaviorism

• Goal is Understanding = Control
• Used Skinner Box for experiments
• Studied Classical conditioning (Pavlov) vs. Operant conditioning (Skinner)

Gestalt Psychology (Mostly Europe)

• Key figures include Wertheimer (1880-1943) and Koffka (1886-1941)
• The whole is different from the sum of its parts
• Examples include Phi-phenomenon and Law of Prägnanz, the Kanisza triangle

Psychological Antecedents

• Structuralism led to Functionalism and Gestalt Psychology
• Functionalism and Gestalt Psychology led to Behaviorism and Cognitive Psychology

Brief History of Cognitive Psychology

• Emergence of Modern Cognitive Psychology happened in 1956 at the MIT symposium
• Ulric Neisser's "Cognitive Psychology" defined the field
• Factors contributing to the rise
  • Disappointment with Behaviorism
  • Linguistics, Noam Chomsky
  • Memory research (Miller, 7+/-2)
  • Developmental psychology; Jean Piaget - object permanence

Cognitive Revolution

• Emerged from Structuralism, Functionalism, Gestalt Psychology, Behaviorism, Linguistics and Computer Science
• This combined to form Cognitive Psychology & Artificial intelligence

Computer Metaphor

• Structures and processes are just like a computer with limited attention and memory capacities
• Knowledge structures are the ingredients and processes are the cooking instructions
• The Information-Processing Approach states a mental process can be compared with the operations of a computer
• A mental process can be interpreted as information progressing through the system in a series of stages, one step at a time in serial processing

Atkinson and Shiffrin (1968)

• Information processing model
• Uses sensory, short-term, and long-term memory stores

Connectionist Approach

• The Parallel Distributed Processing Approach
• Includes connectionism, neural networks and cerebral cortex
• Utilizes parallel processing
• This has inspired current Al and deep learning

The Birth of AI (Russell and Norvig, Chapter 1.3)

• Early days (~1950): Hebbian learning, early neural network approach
• MIT symposium (1956): Newell and Simon with the Logic Theorist
• Early work (till 1969): great expectations, good progress
  • General Problem Solver: mimicking human problem solving
  • LISP: represent general knowledge and reasoning
  • Microworlds: limited problems that could be solved by intelligence (calculus integration, analogies, algebra)
  • Block worlds: manipulating blocks

Challenges to AI in the Sixties

• Initial successes did not scale to more complex problems
• Syntactic manipulations without background knowledge limited progress
• Intractability of problems beyond a few objects / items
• Limitations to basic structures in (early) neural networks or perceptrons
• Knowledge-based systems (1969-1979) were introduced
  • Go beyond general purpose, drop weak methods without domain-specific knowledge
  • Expert systems (e.g. MYCIN) encoded expert knowledge and uncertainty

Return to Neural Networks and the Rise of Data

• Parallel Distributed Processing (Rumelhart and McClelland, 1986) using back-propagation became successful.
• More scientific approaches with statistical approaches and new techniques like Hidden-Markov Models, Data Mining and Bayesian Networks were adopted
• Mid-nineties Agent-based Al became popular with more interest in Human-Level Al and Artificial General Intelligence
• The availability of larger data sets made Al more data-driven, becoming data science
• Data-driven AI like Large Language Models are one step closer to AGI

Definitions of AI (1.1)

• Approaches can be human level or rational with variations on pure and weak AI
  • Rational focuses on math/engineering
  • Human simulates human-like intelligence (empirical)
• The Turing test is acting humanly, GPS is thinking humanly
• Thinking and reasoning relates to acting, which is behavior