Language Learning PDF

Reminder to sign in to AttendanceRadar Quiz is open now: Language Learning  What is language? How do humans learn language? Why don’t animals learn language? Language is insane Stephen King, What Writing Is Ø Language is a system for transfer...

Reminder to sign in to AttendanceRadar Quiz is open now: Language Learning  What is language? How do humans learn language? Why don’t animals learn language? Language is insane Stephen King, What Writing Is Ø Language is a system for transferring complex information between minds, using: marks on paper, squeezing air through your throat and mouth, or moving your hands Language as mental synchronization Measure similarity Speaker Listener Nguyen et al., 2021 Language is insane Ø Learning language requires: Ø Phonetics (producing + perceiving speech) Ø Phonology (sounds/signs used in a language) Ø Morphology (constructing words) Ø Syntax (constructing phrases/sentences) Ø Semantics (mapping meaning to words/sentences) Ø Pragmatics (how language is used) Ø Language learning is possible from sounds, without much explicit teaching of linguistic concepts!! Dinosaur Comics, qwantz.com Dinosaur Comics, qwantz.com Dinosaur Comics, qwantz.com  Where does cognition come from? Nativism Empiricism Key elements of Minds are simply cognition are innate general-purpose We start with a basic pattern detectors outline of which All knowledge of the representations and world comes from our algorithms to use, life experiences which is refined through learning How are (first) languages learned? Two general kinds of views: 1. We have an innate language-specific module, which just needs to be tuned to the specific language we hear 2. We actually learn language “from scratch” Universal Grammar (UG) The UG hypothesis: there is a recently- evolved mental module that Noam Chomsky ØIs innate ØIs specific to humans ØIs used only for language system ØProvides computational mechanisms useful for grammatical processing, such as recursion The girl cried. Mary said that the girl cried. John thinks that Mary said that the girl cried. Arguments in favor of UG Ø “Poverty of the stimulus” argument ØThe speech that children are exposed to is consistent with many possible grammars ØThe fact that they learn the correct grammar implies that they have an innate bias toward that grammar The boy is smiling. The boy who is smiling is happy. Is the boy smiling? Multiple ways we could apply the Rule could be: move rule here (move 1 or both “is”s) “is” to the front 3-5 yo children use the correct rule (which requires parsing), despite little/no experience Arguments in favor of UG Ø “Linguistic Universals” argument ØAll languages share some key features ØThis implies that there some innate UG that constrains what kinds of languages humans create Ø Some proposed Universals: ØLexical categories (noun, verb, adjective, preposition) ØRules of linear order, or word case markings ØAuxiliary (helping) verbs (“may”, “should”) Arguments in favor of UG Ø Emergence of structured languages in environments without structured languages Ø If people spontaneously start using grammatical structure without being taught, it must be innate ØExamples: ØHome sign – deaf children produced signs with consistent grammar without an adult model ØCreole languages – exposed only to impoverished “pidgin” languages, children developed highly- structured combination languages Bickerton, 1983 The Anti-UG View The anti-UG hypothesis: language learning can be accomplished purely via domain-general cognition There is no language-specific module, and language can be learned “from scratch” without innate hints about how it works Arguments against UG Ø Against “Poverty of the Stimulus” Ø Children can pick the correct grammatical rules (with hierarchical structure), solely with a general “Occam’s Razor” preference for simple explanations Ø One approach to proving this: fit flat and hierarchical models to a dataset of child speech The learnability of abstract syntactic principles (Perfors et al. 2011) Ø “Adam” corpus The learnability of abstract syntactic principles (Perfors et al. 2011) Ø “Adam” corpus Ø Try to explain how the adults are speaking using “flat” grammars vs hierarchical grammars The learnability of abstract syntactic principles (Perfors et al. 2011) Ø “Adam” corpus Ø Try to explain how the adults are speaking using “flat” grammars vs hierarchical grammars ØTrying to make a flat grammar work becomes more and more complicated over time Ø Hierarchical grammar solves auxiliary-fronting even having never seen it before Arguments against UG Ø Against “Linguistic Universals”: all supposed “universals” have counter-examples (Evans and Levinson, 2009) ØLexical categories: words in Straits Salish can function as any part of speech The same word is used for “run” and “runner” (additional words and word order resolve the ambiguity) Arguments against UG Ø Against “Linguistic Universals”: all supposed “universals” have counter-examples (Evans and Levinson, 2009) ØLexical categories: words in Straits Salish can function as any part of speech ØLinear orders or markings: neither present in Riau Indonesian ØAuxiliaries: not present in many languages, e.g. Kayardild (Aboriginal Australian)  “Transformer” Architecture Input Representations Output Representations http://jalammar.github.io/illustrated-transformer/ Ø Evidence against needing a pure “grammar” system Ø Syntax is integrated with semantics Ø Evidence against “poverty of the stimulus” Ø Learning succeeds in an unconstrained space Ø Hierarchical structure need not be innate Arguments against UG Ø Modern language models seemingly learn complex language structures without any innate constraints Ø Possible UG response: these models are trained on vastly more data than children 100M words (~12 y.o.) Warstadt et al., 2023 100M words (~12 y.o.) 10M words (~5 y.o.) Warstadt et al., 2023 Why don’t animals use language? Ø Animals can be taught to use “words” to communicate, but seem to lack critical abilities in: Ø Constructing compositional “sentences” Ø Displacement: referencing things not currently present Herb Terrace and “Nim Chimpsky” Why don’t animals use language? Ø Animals can be taught to use “words” to communicate, but seem to lack critical abilities in: Ø Constructing compositional “sentences” Ø Displacement: referencing things not currently present Ø What’s unique in humans? Ø Chomsky: a mutation that allows Universal Grammar Ø Terrace: ability to learn words through shared attention Up next Tomorrow 11:59pm: First paper due Grading rubric is in the assignment on Courseworks Example papers were emailed out on Oct 1st Thursday class: Intro to the brain