Lecture 2: Cognitive Systems PDF

Motivated Learning  Definition: Motivated learning (ML) is pain based motivation, goal creation and learning in embodied agent.  Machine creates abstract goals based on the primitive pain signals.  It receives internal rewards for satisfying its goals (both p...

Motivated Learning  Definition: Motivated learning (ML) is pain based motivation, goal creation and learning in embodied agent.  Machine creates abstract goals based on the primitive pain signals.  It receives internal rewards for satisfying its goals (both primitive and abstract).  ML applies to EI working in a hostile environment.  Various pains and external signals compete for attention.  Attention switching results from competition.  Cognitive perception is aided by winner of competition.  Primitive Goal Creation faucet refill garbage sit on w. can water tank open - + Dual pain Pain Primitive Dry soil level  Reinforcing a proper action  Abstract Goal Hierarchy Sensory pathway Motor pathway  Abstract goals are (perception, sense) (action, reaction) created to reduce tank refill Level III abstract pains and to - + satisfy the primitive goals  A hierarchy of abstract goals is created to satisfy faucet open the lower level goals Level II - + Activation Stimulation Inhibition w. can water Level I Reinforcement - + Echo Need Expectation Primitive Dry soil Level  Goal Creation Experiment in ML MOTOR SENSOR OBJECT REDUCES PAIN INCREASES FUNCTION PAIN Eat Food Hunger Lack of Food Buy Grocery Store Lack of Food Lack of Money Food at Withdraw from Bank Lack of Money Overdrawn Account Account Work The office Overdrawn Account Lack of job in opportunities Study School Lack of job - at opportunities Play Toys - - with  Cognitive Architectures Consciousness and Cognition 5 Janusz A. Starzyk  Goal Creation Experiment in ML Primitive Hunger Pain 1 0 0 100 200 300 400 500 600 Lack of Food 0.5 Pain 0 0 100 200 300 400 500 600 Empty Gorcery 0.5 Pain 0 0 100 200 300 400 500 600 Discrete time Pain signals in CGS simulation Goal Creation Experiment in ML Goal Scatter Plot 40 35 30 25 Goal ID 20 15 10 5 0 0 100 200 300 400 500 600 Discrete time Action scatters in 5 CGS simulations  Goal Creation Experiment in ML Primitive Hunger 0.5 Pain 0 0 100 200 300 400 500 600 Lack of Food 0.2 Pain 0.1 0 0 100 200 300 400 500 600 Empty Gorcery 0.2 Pain 0.1 0 0 100 200 300 400 500 600 Lack of Money 0.2 Pain 0.1 0 0 100 200 300 400 500 600 Lack of JobOpportunitites 0.1 Pain 0.05 0 0 100 200 300 400 500 600 Discrete time The average pain signals in 100 CGS simulations Goal Creation Experiment in ML Comparison between GCS and RL  Compare RL (TDF) and ML (GCS) Mean primitive pain Pp value as a function of the number of iterations: - green line for TDF -blue line for GCS. Primitive pain ratio with pain threshold 0.1  Compare RL (TDF) and ML (GCS)  Comparison of execution time on log-log scale  TD-Falcon green  GCS blue  Combined efficiency of GCS 1000 better than TDF Problem solved Conclusion: embodied intelligence, with motivated learning based on goal creation system, effectively integrates environment modeling and decision making – thus it is poised to cross the chasm Reinforcement Learning Motivated Learning  Single value function  Multiple value functions  Various objectives  One for each goal  Measurable rewards  Internal rewards  Predictable  Unpredictable  Objectives set by designer  Sets its own objectives  Maximizes the reward  Solves minimax problem  Potentially unstable  Always stable  Action depends on the state  Action depends on the states of the environment of the environment and agent  Learning effort increases  Learns better in complex with complexity environment than RL  Always active  Acts when needed http://www.bradfordvts.co.uk/images/goal.jpg  ML goals are build on primitive needs Competing need signals 0.12 Dirty Thirsty 0.1 Drought Threshold Need signal level 0.08 0.06 0.04 0.02 0 0 50 100 150 200 250 300 Iterative step Water Reservoir Abstract Needs Wash in Drink Water Irrigate Water Primitive Needs Dirty Thirsty Drought Abstract needs make its behavior more complex Well Public Money Draw own Spend Money Spend Money Water to Buy to Build Water Reservoir Abstract Needs Wash in Drink Water Irrigate Water Primitive Needs Dirty Thirsty Drought Abstract needs Ground Well Wealthy Tourists' Water Building Taxpayers Attractions Build Water Supply Dig a Well Rise Taxes Ecotourism Well Public Build Water Money Recreation Draw own Spend Money Spend Money Water to Buy to Build Water Reservoir Abstract Needs Wash in Drink Water Irrigate Water Primitive Needs Dirty Thirsty Drought  Environmental Graph of Abstract Goals Management Planning Policy Resource Management Employment Develop Regulate Use Receive Salary and Planning Opportunities Infrastructure Ground Well Wealthy Tourists' Water Building Taxpayers Attractions Build Water Supply Dig a Well Rise Taxes Ecotourism Well Public Build Water Money Recreation Draw own Spend Money Spend Money Water to Buy to Build Water Reservoir Abstract Needs Wash in Drink Water Irrigate Water Primitive Needs Dirty Thirsty Drought  Non-agent Characters’ Actions  All abstract pain neurons have a bias input B that depends on the state of the environment and the preference (bias) of the agent for or against a certain resource or action performed by other actors referred as NACs (Non-agent characters).  The observed bias signal triggers the pain and enforces action. Virtual Agent Using ML  Environmental Graph  Developing Trust In motivated learning, trust is associated with NACs actions. If most of the NAC’s actions are desirable (as when a parent takes care of his child, providing it with warmth and comfort) the ML agent’s trust towards the NAC’s actions increases. Other character features like shyness may be related to overall experience with NAC agents. If most NACs interactions hurt the ML agent it may develop mistrust to all NACs, and become shy. If most NACs run away from the agent it may become fearless. Trust can be computed from where mi number of pain signal related to action nk  Developing Trust In simulation we can show advantage of developing trust Trust values Reward History with Trust vs. without Trust 0.6 0.4 0.4 not trusted NAC with Trust 0.2 trusted NAC with Trust Average normalized reward agent without Trust 0.3 0 0.2 -0.2 -0.4 0.1 With trust Without trust upper limit1 -0.6 lower limit1 0 upper limit2 -0.8 lower limit2 -0.1 -1 0 1000 2000 3000 4000 5000 6000 0 1000 2000 3000 4000 5000 Time Time  Definition of Machine Consciousness Consciousness is attention driven cognitive perception, feelings, emotions, motivations, thoughts, plans, and action monitoring. A machine is conscious IFF besides ability to feel, perceive, act, learn, and remember, it has a working memory (central executive) mechanism that uses attention to focus on selected images or ideas to plan and evaluate the action. It uses all the processes (conscious or subconscious) of the mind; http://hplusmagazine.com/sites/default Photo: www.spectrum.ieee.org/.../biorobot11f-thumb.jpg  Consciousness: functional requirements  Feelings  Emotions  Intelligence  Working memory  Attention and attention switching  Mental saccades  Cognitive perception  Cognitive action control Photo: http://eduspaces.net/csessums/weblog/11712.html http://faculty.virginia.edu/consciousness  Computational Model of Machine Consciousness Episodic Working Planning and Memory Memory & thinking Learning Queuing and Attention Motivation and Action organization switching goal processor monitoring of episodes Episodic Semantic Feelings, emotions, Motor memory memory rewards, and sub- skills cortical processing Sensory Motor processors processors Sensory-motor Data encoders/ decoders Data encoders/ decoders Sensory Motor units units Inspiration: human brain Photo (brain): http://www.scholarpedia.org/article/Neuronal_correlates_of_consciousness  Sensory and Motor Hierarchies  Sensory and motor systems appear to be arranged in hierarchies with information flowing between each level of the sensory and motor hierarchies.  Higher levels provide delayed motor response to perceptions and can overwrite lower-level response.  Feelings and emotions are triggered by the lowest level sensory-motor perception and response.  They are responsible for the primitive pain signals that drive homeostatic behavior. 24  Sensory- Motor Block Semantic Feelings, emotions, Motor memory rewards, and sub- skills cortical processing Sensory Motor processors processors Sensory-motor Data encoders/ decoders Data encoders/ decoders Sensory Motor units units http://www.ourbabynews.com/wp-content  sensory processors are integrated with semantic and episodic memories  motor processors integrated with motor skills  sub-cortical processors integrated with feelings, emotions, rewards and working memory  Working Memory  Platform for the emergence of consciousness  Controls its conscious perception and motor processes  Working memory is integrated with  feelings and emotions  attention and attention switching  perception and learning mechanisms  creation and selection of motivations and higher-level goals http://www.unifesp.br/dpsicobio/eventos/workingmemory/ https://www.simplypsychology.org/working-memory.html  Working Memory Working Planning and Memory thinking Attention Motivation and Action switching goal processor monitoring  Tasks o cognitive perception o attention o attention switching o motivation o goal creation and selection o thoughts o planning o learning, etc. http://prodinstres.pbworks.com  Working Memory Working Planning and Memory thinking Attention Motivation and Action switching goal processor monitorin g  Interacts with other units for o performing their tasks o gathering data o giving directions to other units https:// www.psychedconsult.com/how-to-  No clearly identified decision center help-children-with-working- memory-deficits/  Decisions are influenced by o competing signals representing feelings, motivations, pains, desires, plans, and interrupt signals need not be cognitive or consciously realized o competition is interrupted by attention switching signal  http://www.mukyaa.com Attention Switching !!!  Attention  is a selective process of cognitive perception, action and other cognitive experiences like thoughts, action planning, expectations, dreams  Attention switching  leads to sequences of cognitive experiences  our brains CAN'T do more than one complicated, task at a time http://brandirons.com/ Comic: http://lonewolflibrarian.wordpress.com/2009/08/05/attention-and-distraction-what-are-you-paying-attention-to-08-05-09/  Attention Switching !!!  Dynamic process resulting from competition between motivations sensory inputs internal thoughts http://www.cs.miami.edu https://blog.upsidelearning.com/2010/04/22/multitasking-or-attention-switching/  Attention Switching !!! May be a result of : deliberate cognitive experience (and thus fully conscious) subconscious process (stimulated by internal or external signals) While paying attention is a conscious experience, switching attention does not have to be. https://www.roshancools.com/blog/2014/1/15/jmqbad74gfjjexzfdfd84rmzt2fvc1 Simplified Cognitive Machine Advancement of a goal? Yes No Formulate episode Write to episodic memory Associative memory Saccade control Loop 1 Changing motivation Attention spotlight Action control Changing perception From virtual game Loop 2 Changing environment Visual Saccades What Where C D C D A A B B C D A Input B image Mental Saccades Memory traces in frontal cortex Frontal cortex house Selected part of wife business  the image friends Spotlight on John dog Mental saccade resulting from a visual saccade. Episodic and associative his wife business his house memory network  Perceived input friends John his dog activates object recognition and associated areas saccade Input image of semantic and episodic memory.  This in turn, activates memory traces in the working memory that will be used for mental searches (mental saccades). Mental saccades in a conscious machine Advancement No Attention spotlight of a goal? Yes Loop 1 Learning Mental saccades Yes Changing motivation Continue search? No Loop 3 Plan action? Associative memory Yes Loop 2 No No Action? Changing perception Perceptual saccades Loop 4 Yes Action control Changing environment Loop 5 http://cdn-3.lifehack.org/wp-content  Comprehensive Cognitive Model  Proposed cognitive system organization  Contains  Semantic, episodic and procedural memories.  WTA attention switching  Visual and mental saccades  Scene building  Action planning  And more…  Figure represents our top- level design model  Computational Model: Summary  Feelings and emotions are foundation of goal driven behavior  Self-organizing mechanism of emerging motivations and other signals competing for attention is fundamental for conscious machines.  The working memory controls conscious and subconscious processes driven by perceptions, motivations, thoughts and its attention switching mechanism.  Attention switching is a dynamic process resulting from competition between sensory inputs, motivations and internal thoughts  Mental saccades of the working memory are fundamental for cognitive thinking, attention switching, cognitive planning and action monitoring  Computational Model: Implications  Motivations for actions are distributed o competing signals are generated in various parts of mind  Before a winner is selected, machine does not interpret the meaning of the competing signals  Cognitive processing is predominantly sequential o winner of the internal competition is an instantaneous director of the cognitive thought process, before it is replaced by another winner  Top-down activation for perception, planning, internal thought or motor functions o results in conscious experience decision of what is observed and where is it planning how to respond cognitive motivations and thinking o a train of such experiences constitutes consciousness  NeoAxis Simulation Neoaxis Implementation VIDEO https://www.youtube.com/watch?feature=player_embedded&v=nhXXZgVY67E&hd=1  Google AI and Large Language Models Google produced interesting AI based language models that can pass Turing tests for intelligence 1.PaLM (Pathways Language Model) from Google https://www.youtube.com/watch?v=yi-A0kWXEO4 2.LaMDA (Language Model for Dialogue Applications) https ://www.youtube.com/watch?v=2856XOaUPpg 3.Bard vs. ChatGPT: What's the difference? https://www.techtarget.com/whatis/feature/Bard-vs-ChatGPT- Whats-the-difference Godfather of AI” Geoffrey Hinton warns of the “Existential Threat” of AI https://www.youtube.com/watch?v=Y6Sgp7y178k  Google AI and Large Language Models PaLM (Pathways Language Model) is a 540 billion parameter transformer- based large language model developed by Google AI. PaLM is capable of a wide range of tasks, including commonsense reasoning, arithmetic reasoning, joke explanation, code generation, and translation. OpenAI launched ChatGPT, a chatbot based on the GPT-3 family of large language models (LLM) On December 6, 2023 Google DeepMind introduced Gemini as the successor to LaMDA and PaLM 2. Gemini was designed to be multimodal, meaning it could process multiple types of data simultaneously, including text, images, audio, video, and computer code.  Conclusions 1. Consciousness is computational 2. Intelligent machines can be conscious