NEUR3101 Motor Learning Lectures 8 & 9 PDF

NEUR3101 Motor control Lectures 8 & 9 Motor learning and internals models (from Aplysia to the Brain Machine Interfaces) A/Prof Ingvars Birznieks  Motor learning and memory mechanisms Procedural memory is a part of the long-term memory that is...

NEUR3101 Motor control Lectures 8 & 9 Motor learning and internals models (from Aplysia to the Brain Machine Interfaces) A/Prof Ingvars Birznieks  Motor learning and memory mechanisms Procedural memory is a part of the long-term memory that is responsible for knowing how to perform an action or movement. It is associated with motor skills. Procedural memories can be formed along simple reflex pathways that link sensations to movements. Procedural learning involves learning a motor response (procedure) in reaction to a sensory input. Procedural memory guides the processes we perform and most frequently resides below the level of conscious awareness. This is a type of implicit memory.  Motor learning in simple animals Simple invertebrate nervous systems offer some important experimental advantages: Small nervous systems. The number of neurons in invertebrate nervous systems is small. Large neurons. Many invertebrate neurons are very large, making them easy to study electrophysiologically. Identifiable neurons. Neurons in invertebrates can be classified according to size, location, and electrophysiological properties and can therefore be identified from one individual to another. Identifiable circuits. Identifiable neurons make the same connections with one another in every individual.  Motor learning in simple animals In a pioneering series of experiments beginning in the 1960s, Eric Kandel and his colleagues at Columbia University revealed molecular mechanisms how procedural memory and neural circuits associated with motor learning are formed. For this work Eric Kandel was awarded the 2004 Nobel Prize in Medicine. He studied simple reflexes in Aplysia which served as an excellent model to investigate mechanisms underlying learning. If someone blows gently at your eye, you blink. It is an eye-blink reflex. Similarly, if a jet of water is squirted onto a fleshy region of Aplysia called the siphon, the gill will retract. This is called a gill-withdrawal reflex.  Motor learning in simple animals The gill-withdrawal reflex involves neurons within the abdominal ganglion that can be dissected and studied electrophysiologically. A simple wiring diagram for the gill- withdrawal reflex. The sensory neuron that detects stimuli applied to the skin of the siphon synapses directly on the motor The gill-withdrawal reflex in Aplysia. (a) The neuron that causes the gill to withdraw. mantle is held aside to show the gill in its normal position. (b) The gill retracts when water is sprayed on the siphon. Motor learning in simple animals Like the eye blink, the gill-withdrawal reflex displays habituation after repeated presentation of the water jet. Using this mode one may study habituation sensitisation classical conditioning (associative learning) Habituation Fig. 66.2 Kandel  Unpaired pathway (sensitization) Fig 66.5 Kandel The siphon is stimulated by a light touch and the tail is shocked, but the two stimuli are not paired in time. The tail shock excites facilitatory interneurons that form synapses on the presynaptic terminals of sensory neurons innervating the mantle shelf and siphon. This is the mechanism of sensitization. Long term histological changes Fig 66.8 Kandel  Topographic maps and sensorimotor transformation A topographic map is the ordered projection of a sensory surface, like the retina or the skin, or an effector system, like the musculature, to one or more structures of the central nervous system. Some behaviours in primitive animals follow hard-wired topographic sensorimotor transformation. Example of topographic maps in optic tectum (superior colliculus in mammals). The frog’s optical nerve was crushed and eye rotated by 1800. Axons regenerated and grew back from the part of eye to their original tectal destinations maintaining the same spatial arrangement as before the eye rotation. This caused inadequate behaviour for which motor system could not learn to readjust indicating hard-wired sensorimotor organisation Purves, Fig. 23.7 for this type of behaviour in frogs.  How the CNS uses and learns relationship between activation of single neurons and their contribution in the execution of complex movements? What neural code and corresponding coordinate system is used?  The problem of coordinate system and transcoding in the motor control system  Neuronal population coding of movement direction Georgopoulos et al (1986) showed that individual neurons in the arm area of the primate motor cortex are only broadly tuned to a particular hand reaching direction in three- dimensional space. When individual cells were represented as vectors in their preferred direction and the length of each vector corresponding to their discharge rate the resulting vector sum of all vectors (population vector) was in a direction congruent with the direction of movement. Georgopoulos et al., (1986) Science 233: 1416-1419. Observed Predicted Neuronal population coding of movement direction By recording activity in relatively small population of neurons it is possible to predict parameters of the intended movement. Observed Predicted  What can we conclude? Population directional coding in cortical neurons indicates that action plan is represented in higher level abstract form revealing goal of an action. Lower level commands should be derived from going through multiple transformations converting them into instructions directed to single muscles. Is it a good news for Brain-Machine-Interfaces attempting to use neural activity in the brain to control robotic devices? Yes, as action goal is easier to interpret and convert into movement of any robotic actuator. Remember example in which identical primary motor cortex stimulation could evoke completely different muscle activation patterns after higher signal has passed the processing at spinal circuits and integrated actual sensory information? Can we read the intended action from cortical activity: real-time prediction of hand movement trajectory Wessberg et al., (2000) Real-time prediction of hand trajectory by ensembles of cortical neurons in primates. Nature 408: 361-365.  Instead of real-time movement prediction can monkeys learn to control BMI with their mind? Problems to solve: Brain is a dynamic system and neurons change their association with movement (machine learned patterns are useful only for a short time). Usually we can connect to about 100 nerve cells at the time. Is it enough to control an arbitrary movement? How the brain would know which cells are actually connected to the BMI interface if every mm2 contains hundreds of thousands neurons? is it a good or bad news? or both? BMI is reading the intended action from the cortical activity Monkey is using joystick to move a computer cursor BMI is learning brain activity associated with the movement Monkey is using joystick to move a computer cursor Switching to the BMI controlling the computer cursor Monkey uses joystick to control the cursor as before Associations between joystick and cursor movement weaken in the brain as only neuronal activity controlling BMI matters Currently irrelevant movement associations are extinguished Monkey has learned to control the BMI  Instead of real-time movement prediction can monkeys learn to control BMI with their mind? Yes they can! Brain can! Conclusion: Neural networks in the brain consisting of a small number of neurons can learn to control the robotic devices as their own limbs based on associations with achieved goal regardless of how movement is controlled and implemented. Brain has astonishing ability to associate relationship between single neuron activity and motor outcome. Will it work if a random pool of neurons is selected? Can we find out motor function of selected neurons in a paralysed patient? Carmena et al., (2003) PLoS Biol 1(2): e42.  The mirror neuron system monkey monkey sees does A mirror neuron typically fires both when an animal acts and when the animal observes the same action performed by someone else. Thus, the neural activity pattern of those neurons "mirror" the external (observed) different movement behaviour, as they would themselves control the same task execution of this action. Those neurons fire even when view of a familiar action is obstructed and only cues like sound indicate that action is being performed. Mirror neurons subserve imitation learning. They are found in ventrolateral portion of the no vision premotor cortex. In the BMI context mirror neurons activate when seeing movement without self doing, allowing to identify function of individual neurons even in paralysed person. Figure 17.10 Motor equivalence: learned behaviour may be transferrable possibly even to a machine Motor equivalence is the ability of different motor systems to achieve the same behaviour. For example, writing can be performed using different parts of the body. The circuits in premotor cortex implicated as being available for similar actions produced by different body parts. If you train a skill with one limb there might be the potential to improve performance of that task with other limbs or even mechanical devices and tools. Kandel et al Principles of Neural Science  All good news for the cortical Brain-Machine Interfaces Paralysed person can control Brain-Machine Interface Video -> Hochberg et al., (2006) Nature 442: 164-171. A paralysed person can control Brain-Machine Interface with his/her mind University of Pittsburgh Prosthesis which can feel University of Pittsburgh  End of part I Pause, take a break and come back  Internal Models Internal models represent relationships of the body and the external world. Internal models solve the problem of the coordinate system transformations and transcoding between different stages of the motor planning, execution and control. The INVERSE model converts movement plan (intension) into the motor commands that will produce the intended behavioural goal. The FORWARD model predicts movement outcome and expected sensory consequences (corollary discharge) at any time moment by simulating properties of the effector system from a set of descending commands (efferent copy) and current sensory input. It provides instantaneous movement monitoring as well makes future predictions about movement outcome. The general structure of internal models in the context of the motor control The internal models are theoretical concepts of a process and may be viewed as programs, neural networks or set of computational instructions. They may refer to a different time scales of a motor control, different types of controlled movement outcome and may serve different purposes in feedback and feedforward control loops at different anatomical locations.  FORWARD models A FORWARD model may have multiple functions at different time scale and control levels it may predict and mimic sensory input resulting from an action, it may estimate the current and future state of the body during the movement at different time scales, it may predict movement’s final result, it may coordinate actions of multiple motor control systems predicting and informing what effect action of one system will have on another. Predicting sensory input resulting from the movement and interaction with environment In a popular science it has been extensively discussed that you can’t tickle yourself (Blakemore, Wolpert & Frith, (2000) Neuroreport, 11: 11-16). A self-produced stimulus is perceived as less ticklish than the same stimulus generated externally The sensory receptors that are activated should be the same, but the perception is different Why? When a movement is self-produced its sensory consequences can be accurately predicted and this prediction is used to attenuate predictable sensory effects of a movement not requiring attention. A forward model receives an efference copy from an inverse model, then transforms it into a corollary discharge which at the level of neural circuits cancels the predicted neural signal originating from sensory organs and thus suppresses that part of sensation. The part of the sensory input which is not predicted is not cancelled and might be consciously perceived or might be interpreted as unpredictable deviation from the movement plan and thus may trigger a motor response.  Forward model can provide continuous monitoring of current and future state The Motor system and environment are “noisy” and subjected to multitude of unpredictable influences. The current state of a movement (motor performance) has to be monitored continuously. This is archived by help of a Forward model updating predictions of movement outcome in real-time. Updated information about predicted movement state and its deviation from the movement plan due to unpredictable environmental factors and own miscalculations could be used in feedback and feedforward control loops to correct for those deviations and return to the desired movement plan. For example during pointing movement, if your movement trajectory deviates from the planned, it means that you will miss the target. Forward model will detect this discrepancy and will signal that corrective action is required. If there would be no Forward model, the motor control system would become aware of mismatch in movement performance too late when target would be already missed.  Forward model can provide continuous monitoring of current and future state During fast movement where point-by-point monitoring of the state is crucial like in competitive contact sports the problem is that the motor controller receives sensory information after a certain delay due to slow nerve conduction velocities within nervous system and sensory information processing time. More time delay has to added for motor command appropriate for the received sensory input to reach the effector organ it may become outdated and inadequate for situation which has changed dramatically since sensory input was received and would require a different action. To overcome this, the motor system has to plan action in a simulated future state of the body and environment. This is possible by means of forward models. Forward model can coordinate actions of multiple control systems When an object is held in the hand, the grip force prevents the object from slipping and falling our of hands and is adjusted to the weight of the object and friction between the skin the object. The grip force is precisely controlled so that it is just slightly greater than the minimum grip force needed to prevent such slips. However if you make a swift movement like jumping while object is still in the hand or you are preparing to make a fast movement with hand, you would need to tighten the grip or object would slip due to additional inertial forces. But how would grip force controller (while minding its own business) would know about forthcoming perturbations and would know how much exactly grip has to be tightened? This would be task for a forward model which can predict the arm trajectory, the size of expected load force perturbation, so that corresponding grip force increase necessary to compensate for it can be calculated and communicated to the grip force controller in the advance as reactive control might be too slow.  Consolidation of internal models Computational studies suggest that acquisition of a motor skill involves learning an internal model of the dynamics of the task, which enables the brain to predict and compensate for mechanical behaviour. Various studies have demonstrated that within 6 hours after completion of practice, while performance remains unchanged, the brain engages new regions to perform the motor task - there is a shift from prefrontal regions of the cortex to the premotor, posterior parietal, and cerebellar cortex structures. Note: Remember statement in regard to BMI that cortical representation is dynamic and changes regardless that performed movement is the same. Such change in the neural representation of the internal model may underlie its increased functional stability. During hours that follow completion of practice, representation of the internal model gradually changes, becoming less fragile with respect to the behavioural interference (influences of other similar behaviours).  Consolidation of internal models Slow (long term) learning Fast Slow (long term) Cerebellum Superior Inferior Modified from Kim S et al (2015) PLoS Biol 13(12): e1002312.  Motor skill learning and feedback Knowledge of Results (KR) Information received concerning the extent to which a movement accomplished the intended goal. If you attempt to kick a football through the centre of the goal posts, KR is the information you receive whether your succeeded. Knowledge of Performance (KP) Information received about the state of the movement while being performed. The information provides a basis on which to assess the correctness of the movement. Types of feedback (FB) in the context of motor skill learning Intrinsic feedback - proprioception about the movement process. Extrinsic feedback - information about the result of the movement.  Motor skill learning Accordingly to Paul Fitts and Michael Posner there are three stages to learning a new skill: Cognitive phase - identification and development of the component parts of the movement - involves formation of a mental picture of the movement. In the first stage, movements are slow, inconsistent, and inefficient, and large parts of the movement are controlled consciously. Associative phase - linking the component parts of the movement into a smooth action - involves practicing skilled movement and using feedback to improve it. Movements become more fluid, reliable, and efficient, and some parts of the movement are controlled automatically. Autonomous phase - developing the learned skill so that it becomes automatic - involves little or no conscious thought or attention whilst performing the skill. Movements are accurate, consistent, and efficient. Not all performers will reach this stage. The theory of reinvestment in motor control addresses phenomenon that learned procedures can be disrupted if the performer tries to control them consciously with declarative knowledge. The disruption occurs when learned movement is broken back down into a sequence of smaller, separate, independent units similar to how performance was organized early in learning. Once broken down, each unit must be activated and run separately, which slows performance and, at each transition between units, creates an opportunity for error that was not present in the integrated control structure.  Motor skill learning Closed skills: Environment is predictable and response can be planned - gymnastics, archery, bowling, golf. Movements follow set patterns and have a clear beginning and end. The skills tend to be self-paced, for example a free throw in basketball, and serving in squash or tennis. Open skills: The environment is constantly changing and so movements have to be continually adapted. Environmental contingencies determine the response - soccer, basketball, ice hockey and tennis. Majority of skills will fall somewhere in the continuum between open and closed skill types (Knapp, 1967). During the skill learning, closed skills become more consistent and open skills become more diversified.  Mental rehearsal Mental rehearsal of performing a skill in absence of any overt physical movement. During mental practice largely the same brain areas are active that are active during the physical performance of the task. Can produce large positive effects on performance of a specific task. Evidence has demonstrated that mental rehearsal generates positively transferable motor learning (positive transfer means that practice on one activity results in improvements on another activity). Involves constructing model situations and going through the motions of what to do when and how. Especially beneficial for injured athletes. It is a supplement to physical practice.  The Backwards Brain Bicycle https://www.youtube.com/watch?v=MFzDaBzBlL0 Homework: see “Motor Learning Concept Map.pps” on Moodle Define Learning and Practice Motor Learning Variables Motor Learning Conditions Feedback Procedural Learning Motor skill learning Transfer of Learning Declarative and Associative Learning Motor Learning Occurs Adam’s Schmidt’s In Stages Theory Theory Map of Essential Concepts DM McKeough © 2009