Biopsychology and Learning Past Paper PDF 2023

PSYU2236 Biopsychology and Learning Week 1 Lecture- Introduction to Learning and Classical Conditioning What is learning? Learning is the acquisition of knowledge or skills through experience Typically, learning is revealed by a change in behavioural, physiological, and/or neural react...

PSYU2236 Biopsychology and Learning Week 1 Lecture- Introduction to Learning and Classical Conditioning What is learning? Learning is the acquisition of knowledge or skills through experience Typically, learning is revealed by a change in behavioural, physiological, and/or neural reactions/responses Why is learning important? Learning allows an animal to adapt to their environment Notice important events and learn to ignore events that have no consequence to them (non-associative learning) Learn what stimuli predict events and what behaviours are associated with certain consequences (reward or punishment) to better predict or prepare to seek or avoid consequences (associative learning) Not all changes are a result of learning Behavioural/physiological and or neural changes can occur that are not a result of learning e.g changes that occur due to: ○ Illness ○ Fatigue ○ Drugs How can you tell if a behavioural change is not related to learning? ○ It is not a consequence of experience ○ Often temporary, learning is typically long lasting and can only be changed through experience e.g through extinction or the learning of alternate behaviour Associative vs non associative learning Associative learning: any behaviour in which a new response becomes associated with a particular stimulus ○ This can occur via the creation of associations between: ○ Two stimuli e.g a bell and a food for Pavlov’s dogs ○ A behaviour and its consequence e.g holding out it’s paw and getting a treat Non-associative learning: learning that results in a change in the frequency or amplitude of a behaviour/response after repeated exposures to a single stimulus ○ Increase in response (sensitisation) ○ Decrease in the response (habituation) Non-associative learning: habituation People and animals notice novelty from birth When something new happens, we pay attention to it and show an orienting response  Orienting response: move toward/attend to new event. Look in the direction of and/or lean toward However, after repeated exposure to the stimulus we habituate Habituation is a progressive decrease in response amplitude or frequency as a consequence of repeated experience with a stimulus Habituation is typically stimulus-specific Habituation can be short or long term, lasting days, hours or weeks This typically occurs to a stimulus judged to be little or no importance We engage in this type of learning so we can tune out unimportant stimuli and focus on what matters Long term habituation example: a novel sound initially makes an eating cat panic, but if the sound is repeated daily, the cat habituates and then eats without the slightest reaction Example of habituation: people often habituate to the smell of their own dog on their house or car, or to their own perfume ○ Stimulus specific so people would still notice the pet smell in their friends house of someone else's perfume Habituation is NOT sensory adaptation ○ Sensory adaptation is the tendency of sensory receptors to fatigue and stop responding to an unchanging stimulus- they change their sensitivity to the stimulus ○ This is a physical, bottom up process e.g a dark cinema theatre, 1015 minutes later you can see people in their seats How can I tell if it is habituation or sensory adaptation? With sensory adaptation, you can’t recapture the initial stimulus E.g if you jump into a pool of cold water, after a few minutes you start to adapt. After 15 minutes if you try to get the initial feeling of jumping into the cool water YOU CAN’T (no matter how hard you try) With habituation, you can recapture the stimulus E.g if there is an annoying sound, you can eventually tune it out, but if you attend to it, you can hear it again Habituation and Dishabituation Dishabituation: recovery in responsiveness to an already habituated stimulus Habituation to fridge noise. But the addiction of a novel stimulus reorients the perceiver to the habituated stimulus… dishabituation to fridge noise Dishabituation- recovery in responsiveness to an already habituated stimulus Why does the first bite always taste the best? Habituation Habituation is slower when there is a variety and different foods can act as dishabituation Habituation is stimulus specific (wanting more desert)  Sensory specific satiety Sensitisation An increase in response amplitude or frequency as a consequence of repeated experience with a stimulus Sensitisation is typically not stimulus specific Often occurs when we are anticipating an important stimulus so that we are prepared for important cues Sensitisation (and habituation) are commonly studied using the startle response and orienting response Example is a mosquito sleeping with you… becomes more annoying Desensitisation A decrease in response amplitude or frequency back down to baseline as a consequence of repeated experience with a stimulus Habituation vs Sensitisation When will one occur versus the other? The outcome of the stimulus Results show that the attention elicited by a novel stimulus changes with familiarity ○ Elicited behaviours change over exposures The nature of the change depends on the nature of stimulus Simple stimulus → progressive habituation Complex stimulus → sensitisation → habituation Infant study: after a tone was presented as a dishabituating stimuli along with the chequerboard pattern causing recovery of visual fixation to the pattern When does it occur? Intensity of stimulus ○ Low = habituation ○ High = sensitisation ○ Intermediate = sensitisation then habituation The evolutionary significance of the stimulus can override intensity E.g mother not habituate to childs cru Mozzies carrying diseases etc Dual process theory The underlying processes of habituation and sensitisation can co occur The observable behaviour is the sum of these two processes The habituation effect is observed when the habituation process is greater than the sensitisation process and vice versa Key assumption ○ Response to repeated presentations of a stimulus ref;ect the operation of two separate processes, habituation and sensitisation. Habituation  ○ Produces a stimulus specific decline in responding to repeated stimulation and grows stronger as the number of repetitions increases Sensitisation ○ Process produces increases in responsiveness on early trials, which decay spontaneously over time These two processes are thought to sum to determine overt responding SUMMARY AT THE END OF PART 1 Classical Conditioning 1: Introduction What is classical conditioning? The procedure of repeatedly pairing an initially neutral stimulus (NS) and an unconditioned stimulus (US) - a stimulus that reliably elicits an Unconditioned response (UR) After conditioning the neutral stimulus becomes established as a conditioned stimulus with the capacity to elicit a conditioned response that usually resembles the unconditioned response Key elements NS US UR CS CR Types of US 1. Appetitive US automatically elicits approach responses such as eating, drinking, caressing etc. these responses give satisfaction and pleasure 2. Aversive US such as noise, bitter taste, eclectic shock, painful injections, etc are painful, harmful and elicit avoidance and escape responses Appetitive classical conditioning is slower and requires greater number of acquisition trials, but aversive classical conditioning is established in one, two, or three trials depending on the intensity of the aversive US Stimulus substitution theory Pavlov thought that the CS became a substitute for the US  Innate US-UR reflex pathway CS substitutes for the US in evoking the same response CR and UR produced by same neural region Food → salivation CS → salivation If so, the CR should always be the same as the UR Evidence for stimulus substitution hypothesis Signs tracking (AKA Autoshaping) in pigeons Shows that the nature of the US determines the form of the CR Sign tracking and individual differences Both sign and goal trackers are learning that the CS predicts the food outcome, but only sign trackers ascribe incentive to the cue (CS) Rats who display sign tracking towards a CS predictive of food are more likely to develop compulsive cocaine self administration Research from animals on sign tracking is frequently applied to looking at whether people who overeat or who are susceptible to addiction are overly associated with food, alcohol or drugs Conditioned stimuli that signal reward can become motivational magnets that capture attention, even when individuals are motivated to ignore them Sissgn tracking offers an account of how impulsive and involuntary behaviour begins and is triggered by cues It offers a theory of how addiction gets started, while at the same time, explaining why the erosion of self control induced by sign tracking goes largely unnoticed Evidence against stimulus substitution hypothesis Any study in which the elicited CR is different from the UR This is often the case with aversive US ○ E.g when a tone is paired with a shock, rats will jump to the US (shock) but the CR is typically freezing ○ Freezing is a preparatory defense response Alternative Accounts Preparatory response theory ○ Kimble’s theory proposed that the CR is a response that serves to prepare the organism for the upcoming US Compensatory Response theory ○ One version of the preparatory response theory ○ The compensatory after effects to a US are what come to be elicited by the CS ○ Based on the opponent-process theory of emotion/motivation ○ Central goal is to maintain a state of homeostasis Generalisation: a tendency to respond to stimuli that are similar, but not identical to a CS Stimulus discrimination: the ability to respond differently to similar stimuli- to pick the real deal from the look alikes SUMMARY Week 2- Tutorial Like me hypothesis, bias towards humanness and things like us Mirror neuron system, the same neurons are fired when a monkey picks something up, and when a monkey watches someone pick it up, the same result is found in humans with pain. We physically map the neurons and then the neural pattern activates. Does human animacy matter? Animacy results? Like me hypothesis and mirror neural hypothesis so more likely to learn about people from us. Human > non human. Research reports and tutorials will answer this question. Hand is the same the difference is the belief of whether it is human or not IV- what you manipulate Type of training: Human belief Non human belief A between group design (each participant is one of the IV groups). Within each group, participants train on the same 12 sequences, but with different animach instructions e,g human or non human belief DV- sequence completion time Week 2 Lecture- Classical Conditioning (Part 2): Variations and Limitations Variations on basic classical conditioning A CR without a CS-US pairing Higher order conditioning ○ AKA second order conditioning Sensory preconditioning Higher order conditioning Conditioned responses that involve neutral stimuli Conditioned response is established as a result of a pairing of a neutral stimulus (NS) with a conditioned stimulus (CS) E.g fear of highways after a person travels down one following an accident with a truck Trucks is the first order of response Limitations of higher order conditioning The CR to CS is weaker than to CS1 Approx 50% as strong Higher order conditioning is difficult to accomplish because conditioned inhibition also arises More pairings result in inhibition Sensory preconditioning In second order conditioning, one CS is capable of becoming associated with another CS What if this pairing occurs before they acquire associative strength by pairing one of them with a US A CR to an untrained stimulus can even occur when the association between the CS1 and the second neutral stimulus NS2 is established before the first order conditioned response is learned… this is called sensory preconditioning Real world example sensory preconditioning E.g neighbour (NS1) has a large dog (NS2), one day the dog bites you → dog (CS1) → bite (US). you before afraid (CR) of the dog (CS1) and develop a dislike of your neighbour (NS2) To get the strongest CR: ○ Timing is important- first CS must precede second CS ○ Needs to be contingency (one predicts the other) Can only do a few CS-CS pairings to prevent learned irrelevance ○ I.e metronome → light - so what? Limitations to classical conditioning Overshadowing Blocking Latent inhibition Bright light overshadows the metronome and metronome does not cause a reaction The light previous association with food has blocked learning with new association, because light signalling food, blocks new association Overshadowing can be used to prevent the forming of maladaptive associations E.g to prevent conditioned taste aversions during chemo and or reduce anticipatory nausea Harnessing classical conditioning during chemotherapy to minimise taste aversions. A salient/strong flavoured food is eaten with each chemotherapy session. An association is formed with this taste and chemotherapy induced nausea. This food (stimuli) overshadows the forming of taste aversions with other flavours/food that the individual eats so their enjoyment of their favourite foods is not destroyed by an association with chemotherapy induced nausea Latent inhibition This new pairing is inconsistent with prior experience Familiar stimuli are more difficult to condition a CS than are novel stimuli Real world example of latent inhibition Lecture summary Conditioning of a NS, that is, the development of a CR, can occur without an US ○ Higher order conditioning or secondary conditioning The new NS is associated with a CS and can produce the same CR ○ Sensory preconditioning If a NS is associated with another NS, which then becomes a CS, the first NS also becomes a CS and can produce the same response Limitations in conditioning ○ Overshadowing- after training, the less salient stimulus in a compound NS will fail to become a CS and will not produce the CR ○ Blocking- if a NS is conditioned to become a CS prior to being presented as a compound stimulus, it will serve to block the conditioning of the NS it is presenting with  ○ Latent inhibition- if a NS is presented without the US prior to the CS→US conditioning training, conditioning is impaired Classical Conditioning Part 3: Timing, Extinction, and Inhibition What factors determine whether and how much classical conditioning occurs? Determinants of classical conditioning ○ Intensity of US ○ Excitatory vs inhibitory CS ○ Time relationship between stimuli Simultaneous conditioning Delayed conditioning Tract conditioning Backward conditioning ○ Acquisition → extinction → spontaneous recovery Intensity of stimuli Stimulus influences the course of both appetitive and aversive classical conditioning More intense CS are more effective in accelerating the acquisition of a CR ○ Fewer trials needed More intense US are especially powerful ○ Extremely intense aversive US can produce PTSD Excitatory vs inhibitory stimuli To make our environment more predictable we search for relationships between stimuli Some stimuli are predictive of the occurrence of a stimulus and some are predictive of its absence Both stimuli are just as important to learn about Excitatory When the CS products the occurrence of the US it becomes an excitatory CS CS has a positive relationship with the US ○ The CS acquires an ability to excite the organisms ○ The CS activates behavioural and neural responses related to the US in the absence of the actual presentation of the US Inhibitory conditioned stimuli When the CS predicts the absence of the US it will become a conditioned inhibitory stimulus (CS-) ○ CS has a negative relationship with the US Why predict the absence of something ○ We function best in a predictable environment ○ In bad environments we need to know when we are safe- so anxiety levels cna drop ○ In good environments we need to know when to seek alternatives ○ Trackwork on train line select alternative response bus ○ Sign is an inhibitory stimulus so it inhibits us not to sit on a bench for hours Temporal relationships between pairing a. Simultaneous conditioning: CS and US are presented and terminated together b. Delayed conditioning: CS is presented alone for a while, then the US is presented and the CS and US typically terminate together. This is the most effective way c. Trace conditioning: CS begins and ends before US is presented d. Backward conditioning: CS is presented after US is terminated What happens when there is a long delay? ○ Onset of CS precedes US by at least several seconds ○ CS continues until US presented ○ Like trace, US-CS interval impacts conditioning but not as severe, if you delay food after 10 seconds from bell then the salivating starts 8-9 seconds after bell Backward conditioning ○ Level markedly lower ○ Order is important Reducing, eliminating or inhibiting a behaviour Sometimes we need to decelerate a behaviour, through punishment, extinction and inhibition Extinction Elimates CS Repeated present the CS without the US With repeated exposure to the CS it stops being a preductor of the US and the CR decreases ans stops  Systematic desensitisation and exposure therapy Presents stimulus in neutral environment Acquisition, extinction and spontaneous recovery Extinction occurs gradually, after extinction there is no CR Relearning- fewer trials are needed to get back to CS US pairing Number of trials does not determine rate of extinction The total duration of exposure to the CS determines how fast the CR is extinguished The impact of the duration, rather than the number of times an individual is exposed to the CS, an important consideration for exposure therapy Pavlov’s eureka moment External inhibition- the context where the dog learnt the CR has been changed, so now it inhibits the CR Conditioned inhibition Conditioned in which a NS is associated with the presentation of a US is known as excitatory conditioning The result is that he CS, now called an excitatory CS or CS+ acquires the capacity to regularly elicit a CR Conditioning in which the NS is associated with the absence or removal of a US is known as inhibitory conditioning Result is that he CCS now called an inhibitory CS, conditioned inhibitor or CS-, comes to inhibit the occurrence of the CR  Signs that say closed or out of order are meaningful because we don't normally use it. Normal state of play is that there is an absence of something we would usually expect to see Week 3 Lecture- Classical Conditioning 4- Contingency and The Rescorla-Wagner Model Contiguity First principle identified by associationists and refers to the connectedness in time and space of two items The more closely together (contiguous) in space and time the two items, the more likely they are associated As delay increases between CS and US, the rate of learning decreases Last two, very little learning happens but they are highly contiguous Contiguity is important but not sufficient Kamin’s seminal study on blocking also showed that contiguity alone wasn't sufficient Homework- explain why blocking isn't sufficient for contiguity to learn Contiguity is necessary for classical conditioning … but simultaneous and backward conditioning and the phenomena of blocking demonstrate it is not sufficient Must be relationship between the CS and the US to occur Contingency Refers to the predictability of the occurrence of one stimulus from the presence of another Increasing delay between the CS and the US results in the CS becoming less useful as a predictor of the US Contingency is the probabilistic relationship between the US and given that a CS has occurred The probabilistic relationship is called Wagon R model Contingency Theory  A CR develops when the conditioned stimulus CS is able to predict the occurrence of the unconditioned stimulus US After repeated CS US pairings, the individual can begin to predict when the US is coming based on the CS being present When the CS is presented, the individual forms an expectancy of the US, this expectancy is what drives the CR Role of expectations and surprise CS → expectations → get what you expected → no change needed → no learning CS → expectations → surprise you get more → increases your expectations → learning (can work the same with decrease) Contingency depends in reliability and uniqueness Reliability: how often is the CS followed by US, what is the probability that the US will occur after the CS has occurred Uniqueness of CS-US pairing: how often does the US happen without CS? What is the probability of the US occurring given that no CS has occurred Quantifying contingency relations Predictive relationship between CS’s adn US’s The CS has to convey information about US occurrence p(US/CS) > p(US/no CS) The probability of a US occurring given that a CS is present Is greater than The probability of a US given that no CS is present If this condition is met, learning (excitatory conditioning) will occur Left side of equation notes the percentage of CS that are temporarily contiguous (paired) with a US If p = 1.0 then 100% of CS are paired with the US If p = 0.5 then 50% CS’s are paired with US’s and 50% are presented alone Right side of equation notes the percentage/proportion of time intervals without a CS in which a US occurs If p = 1.0 then US’s are presented on 100% of the time intervals with no CS present If p = 0.5 then the US’s are presented 50% of the time with no CS present For both groups, there is a 40% chance that bells will be followed by a shock, however for group B, shock is less likely when no bell is sounded, and, for this group, the bell becomes a fearful stimulus Interim summary When subjects experience CS and US’s tat are positively correlated, they acquire a conditioned response to the CS; this is called excitatory conditioning When subjects experience CS and US that are negatively correlated, responses are inhibited (not performed) when the CS is present; this is called inhibitory conditioning or conditioned inhibition When subjects experience CS’s and US’s that are not correlated there is no conditioning It can be determined if learning is likely to occur by quantifying and comparing the probabilities of the US occurring with and without the CS Rescorla’s classic conditioning experiment Study provided ground work for the seminal rescorla-wagner model of conditioning Basic method: Conditioned emotional response (CER) procedure with rats: Phase 1: operant conditioning to establish steady bar pressing Phase 2: classical conditioning to establish CER Procedure CER training (daily for 5 days) All rate exposed to 12 tones Tones were 2 mins long and mean inter tone interval was 8 mmin Probability of shock (the US) during tone was 0.4 for all rats Groups differed in probability of shock during the inter tone interval During the inter tone interval: Group 0: no shocks Group 1: shocked probability of 0.1 Group 2: shocked probability of 0.2  Group 4: shocked probability of 0.4 Remember: the probability of the shock occurring with a tone was 0.4 for all the rats Phase 3: After CER training, the rats returned to the bar pressing for food While the rats were bar pressing, the tones were presented as before, but no shocks were given If the tone triggers the conditioned fear response e.g freezing, it interferes with (suppresses) the expression of the operant conditioned response (bar pressing) Conditioning was assessed by a suppression ratio (the lower the ratio, the greater the suppression, the stronger the conditioning) Summary of results Whenever p(US/CS) > p(US/no CS): CS is an excitatory CS That is; CS predicts US Amount of learning depends on size difference between p(US/CS) and p(US/no CS) Whenever p(US/CS) > p(US/no CS): CS is an inhibitory CS That is; CS predicts ABSENCE of US The amount of learning depends on size differences between p(US/CS) and p(US/no CS) Part 1 summary: Learning occurs when there is a prediction error I.e discrepancy between the actual outcome of a conditioning trial and the expected outcome of that trial The Rescorla-Wagner model allows us to quantify the size of the contingency relationship between a CS and a US Part 2: The Rescorla-Wagner Model Mathematical model designed to predict the outcome of classical conditioning procedures on a trial by trial basis Learning will occur only when the subject is surprised, that is, when what actually happens is different from what the subject expected to happen Always three possibilities in conditioning trials: ○ Excitatory conditioning ○ Inhibitory conditioning ○ No conditioning at all Which of these three possibilities actually occurs depends upon: ○ The strength of the subjects expectations of what will occur ○ The strength of the US that is actually presented Six rules of the Rescorla-Wagner model 1. If the strength of the actual US is greater than the strength of the subjects expectation, all CS’s that were paired will receive excitatory conditioning 2. If the strength of the actual US is less than the strength of the subjects expectation, all of the CS’s that were paired with the US will receive some inhibitory conditioning 3. If the strength of the actual US is equal to the strength of the subjects expectation, there will be no conditioning 4. The larger the discrepancy between the strength of the expectation and the strength of the US, the greater the conditioning that occurs (either excitatory or inhibitory) 5. More salient CS’s will condition faster 6. If two or more CS’s are presented together, the subjects expectation will be equal to their total strength (with excitatory and inhibitory stimuli tending to cancel each other out) Rescorla-Wagner: Acquisition With each successive conditioning trial, the expectation of the US following the CS should get stronger, and so the difference between the strength of the expectation and the strength of the US gets smaller Therefore, the fastest growth in excitatory conditioning occurs in the first trial, and there’s less and less additional conditioning as the trials proceed When the CS elicits an expectation that is as strong as the USm the asymptote of learning is reached, and no further excitatory conditioning will occur with any additional CS-US pairings SHIFT COMMAND 4 SLIDE 43 Is learning really asymptoting? The R-W model says no learning is occurring after asymptote reached… but it is important to remember that this asymptote reflects the maxing out of the behavioural performance/response that can be measured. Learning might still be occurring! Rescorla-Wagner: Blocking When two CS’s are presented, the subjects expectation is based on the total expectations of both No conditioning occurs to the added CS because there is no surprise- the strength of he subject’s expectation matches the strength of the US Increasing the strength/size of the US when presenting the compound CS may prevent blocking effects Rescorla-Wagner: Extinction & Conditioned Inhibition The strength of the expected US is greater than that of the actual US, leading to a decrease in the association between the CS and the US ○ Following acquisition CS → No-US This leads to any CS or compound CS to acquire some inhibitory conditioning ○ CS → No-CS learning occurs until the absence of the US is no longer surprising  ○ CS → conditioned inhibitor R-W: Combining CS’s When two CSs that elicit CRs on their own are combined, the expected US of the compound CS is roughly equal to their total strength E.g a CS+ and a CS- conditioned separately are presented as a compound CS and offset each other R-W Overexpectation effect The R-W predicted the overexpectation effect There is a decline in responding to a pair of well established conditioned stimuli (CS’s) when they are presented as a compound Overexpectation effect:if two CS’s are conditioned separately and then together, the animal has am over-expectation about the size of the US , and both CS’s will experience some inhibitory conditioning ○ The animal expects twice of the US, so if only the same US follows the compound CS as did each CS alone, this does not meet the animals expectations SHIFT COMMAND 4 REST OF SLIDE 48 The frequency principle of CS-US pairings would predict a stronger CR to the compound stimulus ○ I.e the mire trials linking the CS and US, the stronger the CR It would not be expected that more CS-US pairings would result in a weakening of CS-US associations, yet this occurs in conditions like these where compound CS’s experience inhibitory conditioning due to the overexpectation effect Summary Can be thought as a model of US effectiveness Unpredicted (surprising) US is effective in promoting learning Well predicted US is ineffective in promoting learning Alternative Theories of Classical Conditioning Rescorla-Wagner model focuses on the US side of the association ○ Unpredicted (surprising) US is effective in promoting learning ○ Well-predicted US is ineffective in promoting learning Alternative theories focus on attention to the CS instead A common feature of these theories is the assumption that the learner will pay attention to informative CS’s but not uninformative CS’s These theories might also be called theories of CS effectiveness, because they assume that the conditionability of a CS< not the effectiveness of the US, changes from one situation to another R-W Cannot Explain Latent Inhibition  Recap on latent inhibition/pre-exposure effect: Classical conditioning proceeds more slowly if a CS is repeatedly presented by itself before it is paired with the US The Rescorla-Wagner model does not predict this CS pre-exposure effect When the (to-be-CS) stimulus is presented by itself before the training, the expected US is zero, and the actual US is zero According to the Rescorla-Wagner model, since the actual US = expected US, there should be no learning of any kind during the pre-exposure However, subjects evidently do learn something during the pre-exposure trials, which reduces their ability to later develop a CS-US association Attention-based theories are able to explain the pre-exposure effect ○ Because the CS predicts nothing during the pre-exposure, attention to the CS decreases, and so conditioning is slower when the CS is first paired with the US at the beginning of the conditioning phase Comparator Theories of Conditioning Comparator theories assume that the animal compares two likelihoods: The likelihood that the US will occur in the presence of the CS The likelihood that the US will occur in the absence of the CS Comparator theories are similar in many ways to Rescorla-Wagner and Attentional theories, but differ in two ways: They do not make predictions on a trial-by-trial basis ○ 1. They assume what is important is not the events of individual trials but rather the overall long-term correlation between a CS and the US ○ 2. They propose that the correlation between CS and US does not affect the learning of a CR but rather its performance E.g. Subjects may learn an association between a CS and a US that cannot initially be seen in their performance, but this learning can be unmasked if the strength of a competing CS is weakened How might we unmask learning in blocking? Comparator theories suggest that learning still occurs to a blocked stimulus, it is just masked…. What if you extinguish the blocking CS? Would any conditioning to the blocked CS be revealed? Blocking revisited SHIFT COMMAND 4 SLIDE 57 Comparator theories of learning Comparator theories differ from R-W’s model in two ways: ○ 1) Unlike R-W, comparator theories assume that both the CS and contextual stimuli can acquired equal excitatory strengths, because both have been paired with the US  ○ 2) Comparator theories also assume that a CS will not elicit a CR unless it has greater excitatory strength than the contextual stimuli The animal has learned something about the CS: that the US sometimes occurs in its presence, but the animal will not respond to the CS unless it is a better predictor of the US than the context Context plays a critical role in determining the response Rescorla-Wagner, Attentional and Comparator Theories: Common Themes The predictiveness or informativeness of the stimulus is a critical determinant of whether a CR will occur The predictiveness or informativeness of a stimulus cannot be judged in isolation It must be compared to the predictiveness of other stimuli also present in the learner's environment. ○ In reality, contextual cues are present when the US is absent as well as present à reduces the uniqueness of the relationship with the US and therefore limits learning Summary Unblocking doesn’t sit well with the Rescorla-Wagner model – but there are other theories that offer alternative accounts of the learning process Theories of attention – Mackintosh (1975) Pearce and Hall (1980) focus on how much attention the CS is able to attract, as opposed to how much associative strength a US can support (Rescorla-Wagner, 1972). Comparator theories take contextual cues into account But R-W model is important, and still influential, because it still has a lot of explanatory power and was first in terms of developing a cognitive explanation of classical conditioning processes Week 4- Operant Conditioning Part 1 Classical- pairing by timing Operant- consequence for behaviour Skinner box- the organism operates on its environment to achieve some desirable outcome Some behaviour that can be done to obtain reward ○ Rate measured by experimenter A dispenser of food is used as reinforcer (reward) Tones or lights to signal availability of opportunity for reward or pending punishment ○ Used in discrimination and generalisation studies E.g measure rate of bell rings to gain food (consequence) Probability increases as positive reinforcement Shaping or successive approximations  Shaping is use of reinforcement of successive approximations of a desired behaviour When using a shaping technique, each approximate desired behavior that is demonstrated is reinfroes, while behaviours that are not approximations of the desired behaviour are not as reinforced E.g cats owner first rewarded when they lifted their paw, then when they reach towards the bell, then when they rang the bell etc The Behaviour Increases Decreases Added to the environment Positive reinforcement Positive punishment Removed from the Negative reinforcement Negative punishment environment (avoidance learning) Positive Reinforcement Behaviour + consequence = outcome Texting in class + more interesting than lecture = keep on texting Drinking coffee one morning + feeling more awake = coffee every morning Positive Punishment Adding something to the environment that causes behaviour to decrease Made to do chores = something added to make it bad Negative Reinforcement Something is removed from the environment that causes the behaviour to increase, so that something must have been unpleasant E.g you got caught in traffic and missed the meeting, found out the meeting was important and it will increase the likelihood of you being late again Applying sunscreen to prevent sunburn Strengthen a behaviour that avoids or removes a negative outcome Eating healthy foods to avoid being sick Anxiety engaging in avoidance techniques which means they will probably keep doing it Negative Punishment Something is removed from the environment that causes behaviour to decrease in frequency, that something must have been pleasant Aka response cost or omission training, involve a removal of a stimulus following targeted behaviour, that person values or enjoys or desires To facilitate the process they may be reinforced for exhibiting another more desirable behaviour (DRO: differential reinforcement of other behaviour) E.g if you don’t tidy your room you will lose your phone Behaviour-Consequence Relationships: Emotions Schedules of Learning Continuous schedule Behaviour is followed by a consequence each time it occurs Excellent for getting a new behaviour started Behaviour stops quickly when reinforcement stops Not the best Thinning intermittent reinforcement One of two common ○ Increase response ratio or the duration of the time interval between response → reinforcer Providing instructions such as rules and signs e.g spanish crossing has countdown Partial schedules for resistance to extinction Ratio- after predetermined number of responses you get outcome Interval schedules- the first response after the time has elapsed → outcome e.g every minute no matter how often you respond relies on TIME after 1 Fixed schedule- every 5 responses (ratio) or every 5 minutes (interval) outcome e.g a predictable schedule Variable schedules e.g every 2-5 minutes or every 2-5 responses e.g unpredictable schedule Can combine Fixed ratio Variable ratio  Fixed interval Variable interval Fixed Ratio Behaviour/reinforcement 100/1 or 15/1 Response rate: higher ratio = faster responding Behaviour: tend to work hard (ratio run) receive reinforcement, then brief post reinforcement pause- see point A on figure, then work hard RESISTANCE TO EXTINCTION: LOW Behaviour on a fixed ratio High rates of responding → pause after receiving rewards (PRP) then onwards for the next reward Make the number of responses too high → ratio strain ○ A disruption in responding due to an overly demanding response requirement Ratio strain: Avoidance, aggression and unpredictable pause in responding e.g if things don't get better then I don't know how much longer I can do this E.g pressing a lever for some food is fine but doing backflips isn't worth my time Over time they can predict when they know it is going to happen E.g working harder just before you get the reward Variable Ratio Schedule Behaviour / reinforcement: random or unpredictable number of responses between reinforcements Response rate: fast Behaviour: work hard and at a steady rate RESISTANCE TO EXTINCTION: HIGH  What if the rewards came on a temporal basis? How would it affect the rate of responding? ○ Reduce it- no point working it its not making the rewards come any faster ○ E.g button pressing on a sydney pedestrian crossing is on a fixed interval Fixed Interval Schedule Behaviour or reinforcement after 1st then fixed amount of time Response: scalloped Behaviour: high before reinforcement and a long pause after Lowest rate of responding RESISTANCE TO EXTINCTION: LOW Variable Interval Schedule Behaviour / reinforcement first response then “average” time period elapsed Response rate is slow Works at a steady rate RESISTANCE TO EXTINCTION: HIGH  E.g petrol prices follow a variable interval schedule of reinforcement Highest rate of responding Variable ratio Fixed ratio Ratio → consistent rates of responding Variable interval schedule Fixed interval schedule Part 1 Summary Operant conditioning a relationship between a behaviour and its consequence is learned ○ + and - reinforcers increase the likelihood of that behaviour ○ + and - punishment decrease the likelihood of that behaviour Different schedules of reinforcement can be used to produce different patterns of behaviours that are more or less resistant to extinction Part 2: Changing Behaviour Differential Reinforcement Four procedures that incorporate reinforcement to address and treat disruptive behaviour are: 1. Differential reinforcement of other behaviour (DRO) 2. Differential reinforcement of low rates of responding (DRL) 3. Differential reinforcement of incompatible behaviour (DRI) 4. Differential reinforcement of alternative behaviour (DRA) not necessarily incompatible Differential reinforcement of other behaviours Subject periodically receives positive reinforcer provided it is engaged in other behaviours  ○ E.g sitting quietly (thus omission training) involved reinforcing other behaviour, whole interval DRO (WIDRO)- if target behaviour has not occurred throughout entire period e.g 1 minute → positive reinforcere for other behaviour Differential reinforcement of low rates of responding Teacher wants the child to wash his hands, but not more than once before lunch. Using the DRL, they get to be first in line to lunch if he avoids washing it more than once Differential reinforcement of incompatible behaviour (DRI) A teacher wants the child to remain in his seat so every time he gets out, the behaviour is ignored but when he stays in, he gets rewarded with a sticker Differential reinforcement of alternative behaviour Every time a child makes a demand, parent ignore him, when the child asks politely they listen Establishing operations EO’s are factors that affect the effectiveness of reinforces, the intensity, amount, and type i.e quality of reinforcer determines its effectiveness Reinforcer magnitude Larger the reward the faster the acquisition of learning Reinforcer must be sufficient magnitude for it to be worth making the response If income is less than childcare costs, parents won't return to work Reward magnitude is often a matter of being in the eyes of the beholder E.g rats run faster for the same amount of food but run faster for the food when its broken into more pieces A large reinforcer to a 5 year old may be a weak reinforcer to a 25 year old A few small reinforces are stronger than 1 large one Contrast effects Shifting the value of the reward in mid stream is also effective in changing behaviour known as contrast effects Reinforcer magnitude is all a matter of relativity Contrast effects are obtained when the quality of the reinforcer is switched as well Responding is influenced by the reinforcement characteristics that an organism has come to expect in its past Delay of Reinforcement Gradient of delay: the delay decreases the contiguity between response and outcome ○ Temporal contiguity is an important factor in the effectiveness of operant conditioning. Golden retriever obedience training will be more effective if he is rewarded straight after trick The delay decreases the contiguity between response and outcome ○ Long delays make it difficult for the person/animal to see the relationship between their response and the consequence. ○ A delay allows time for other behaviours to occur during the interval → superstitious reinforcement of them. ○ Deleterious effects of delay can be reduced by providing a signal that the reward is coming i.e. clicker. Contiguity and Reinforcement SHIFT COMMAND 4 SLIDE 48 SHIFT COMMAND 4 SLIDE 29 Gradient of Delay Delay of reinforcement gradient: The effectiveness of the reinforcement decreases as the delay increases When there is no bridging cue, just how long can the interval be? 30 seconds and the rat can still learn the association Speed of Reward Addiction is linked to the speed of reward ○ Why checking emails is more rewarding than finishing a psych report This is why Internet pornography is much more addictive than pornography contained in a magazine or video. And this is exactly why modern poker machines are much more addictive than older pokies – the “one-armed bandits”. Modern pokies increase the gambling 'dosage' to much higher levels. All this speed means more bets, and more bets mean more excitement and more excitement means more dopamine.  More dopamine means the modern pokie is more addictive than its predecessors Response-Reinforcer Contingency The Reinforcer must be the result of some Response ○ The greater the consistency between the Reinforcer and the Response, the quicker/more effective the conditioning. GOALS MUST BE SET AND MET BEFORE A REWARD IS GIVEN Response Control: two month old infants who can make a mobile move, smile and coo at it, while those who have no control over its motion, stop smiling Reinforcers: Primary and secondary A primary reinforcer is a stimulus that is reinforcing even without previous training. Primary reinforcers are biologically relevant stimuli or events i.e. they have survival value. Examples include food, water, and sex. A conditioned (secondary) reinforcer is an arbitrary event (such as a tone, clicker or token) that increases the frequency of an operant response. Events that have been associated with rewarding experiences acquire reinforcing power. They are reinforcing because they permit an organism to obtain a primary reinforcer. There are a variety of classes of reinforcers that can be subsumed under the heading secondary reinforcer: ○ Tokens: money ○ Social: smile or thank you → feel good ○ Activity reinforcers: screen time (premack principle) ○ Covert reinforcers: giving yourself a quiet slap on the back for sticking to your goals Functions of Conditioned Reinforcement Conditioned reinforcers: Tell organism it has done right thing Tell the organism what to do next Bridge long periods between unconditioned reinforcers Conditioned Reinforcement: Clicker training Used for training animals – from dogs to horses to dolphins (Karen Pryor) Pair a hand-held clicker with food through straightforward classical conditioning. The sound of the clicker can then reinforce other behaviours – such as “Sit”. Clicker Training: Advantages Clickers sound the same no matter how you are feeling when you press it A clicker is easier to discriminate from everything else we say to the dogs! Split second timing is possible with the clicker thereby reinforcing the precise behaviour. Using a primary reinforcer, such as food, can cause the dog to become focused on the food, and the food giver, rather than on the behaviour. The clicker can reinforce the behaviour immediately. So how powerful is a secondary reinforcer? A number of variables affect the strength of a secondary reinforcer: 1. The magnitude of the primary reinforcer 2. The number of pairings (with the primary reinforcer) 3. Time elapsing between the presentation of the secondary reinforcer and the primary reinforcer Token Economies Token Economies were pioneered by Ted Allyon and Nate Azrin Used to teach and maintain normal behaviour of psychotic residents in a psychiatric institution. Were suffering from severe problems with verbal and social behaviour Those included in the token economy earned little metal tokens by making responses. Use of Tokens with Humans Reinforcement in a token economy. This graph shows the effects of using tokens to reward socially desirable behaviour in a mental hospital ward. Desirable behaviour was defined as cleaning, bed making, attending therapy sessions, and so forth. Tokens earned could be exchanged for basic amenities such as meals, snacks, coffee, game-room privileges, or weekend passes. The graph shows more than 24 hours per day because it represents the total number of hours of desirable behaviour performed by all patients in the ward. SHIFT COMMAND 4 GRAPH SLIDE 60 Activity as a secondary reinforcer: Premack’s Theory "What is a reinforcer?" David Premack (1965) supplied a way out of the dilemma--just look at what an organism does as what is important "reinforcement involves a relation, typically between two responses, one that is being reinforced and another that is responsible for the reinforcement. This leads to the following generalisation: Of any two responses, the more probable response will reinforce the less probable one" (1965, p. 132). This generalisation, known as the Premack Principle, is usually stated somewhat more simply: High probability behaviour reinforces low probability behaviour Premack’s Theory of Reinforcement If you eat your veggies → then you get to eat ice cream If you study first then you can watch tv And punishment If you’re naughty you have to do chores Chaining Chaining refers to a method of teaching a behaviour using behaviour chains. Behaviour chains are sequences of individual behaviours that when linked together form a terminal behaviour. It involves reinforcing individual responses occurring in a sequence to form a complex behaviour. It is frequently used for training behavioural sequences (or "chains") that are beyond the current repertoire of the learner. The chain of responses is broken down into small steps using task analysis. Parts of a chain are referred to as links. Problem: rat needs to get to top platform ○ Response chain: a sequence of behaviours occurring in a specific order reinforced on the occurrence of the terminal response. ○ Each step in the response chain acts both as a conditioned reinforcer (SR) for the previous step and as a discriminative stimulus (SD) for the next step Discriminative Stimulus Discriminative stimulus: a stimulus that indicates whether or not responding will lead to reinforcement. ○ The ladder serves as a discriminative stimulus indicating that if they climb up they will be rewarded (or will be closer to the reward) SHIFT COMMAND 4 SLIDE 67 Two main chaining techniques Forward Chaining: Using forward chaining, the behaviour is taught in its naturally occurring order. Each step of the sequence is taught and reinforced when completed correctly. Once 1st is mastered →next step Backward Chaining: Using backward chaining the learner first performs the final behaviour in the sequence at the predetermined criterion level, reinforcement is delivered. Next, reinforcement is delivered when the last and the next-to-last behaviours in the sequence are performed to criterion. ○ This sequence proceeds backwards through the chain until all the steps in the task analysis have been introduced in reverse order and practised cumulatively Both techniques more successful than whole task learning Response Chains Response Chain Considerations Dependent on reinforcement for continued performance. If a link breaks, all behaviours prior to the broken link will be extinguished. Each reinforcer does not have equal value. Responses farthest from reinforcement are the weakest and easiest to extinguish. Mission Impossible: Squirrel chaining behaviour This takes place in England - the owners of the yard added each piece of the Rube Goldberg contraption slowly so that when the squirrel learned one section and got the nuts, they then added the next section. Finally it ended with what you see on the clip! It took over 2 weeks to get to this point. Part 2 Summary Different reinforcement schedules can also be used to reduce unwanted behaviours ○ The undesirable behaviour is not reinforced, but instead other, incompatible, or alternative responses are reinforced ○ Or lower rates of the unwanted behaviour targeted are reinforced. The magnitude, delay, and contingency of the reinforcer impacts is effectiveness Reinforcers can either be primary (e.g., food) or secondary (money) Premack’s principles proposed that activities are reinforcing ○ High probability behaviours (behaviours that the organism frequently does when given the opportunity) reinforce low probability behaviours (behaviours that the organism would be unlikely to do on their own) Complex chains of behaviours can be learned through a process of chaining where separate “chains” or behaviours are learned and linked together. Week 5- Extinction in Operant and Classical Conditioning What is learning? Change in behaviour as a result of experience ○ Adaptive: behaviour is plastic and flexible to suit the environment in which we find ourselves Non associative learning ○ Habituation, sensitisation- changes to make our experience more suited to the surroundings Associative learning ○ Learning to predict relationships between events, actions and outcomes ○ Allows us to seek out and obtain rewards and avoid situations that are dangerous Mechanisms of Extinction Pavlovian conditioning- learning that an event/object/context predicts a particular outcome US: a biologically salient event/outcome UR: the natural response elicited by the US CS: an initially neutral stimulus that, through repeated pairings with the US, comes to elicit the same response CR: the response that is elicited by the CS (usually the same as the UR) Operant conditioning- learning that particular actions will result in particular outcomes  Reinforcement- an event that increases the likelihood of repeating the response that preceded it Positive reinforcement = presentation of a favourable (appetitive) outcome Negative reinforcement = outcome is removal of an aversive stimulus Punishment = an event that decreases the behaviour that precedes it Positive punishment = adding an aversive outcome Negative punishment = outcome is removal of an appetitive stimulus Prediction error The amount of learning that will occur in any one trial depends on how surprising the unconditioned stimulus is (Rescorla and Wagner, 1972) The difference between the expected and the actual outcome ∆Ⅴ=α(λ−Ⅴ) ∆Ⅴ = change in associative strength α = associability or learning parameter λ = magnitude of the US Ⅴ = current associative strength What happens if the expected outcome is omitted? Negative prediction error: I expect something to happen and it doesn’t Pavlovian Conditioning PAVLOVIAN CONDITIONING = learning that an event / object / context predicts a particular outcome Appetitive: Unconditioned Stimulus is favourable E.g. Pavlovian conditioned approach Aversive: Unconditioned Stimulus is unfavourable E.g. Pavlovian fear conditioning EXTINCTION = decrease in the conditioned response as a result of CS-alone presentations Theories of Extinction Unlearning or erasure of the CS-US association New learning of a second, competing association that inhibits the expression of the original association SHIFT COMMAND 4 SLIDE 12 SHIFT COMMAND 4 SLIDE 13 SHIFT COMMAND 4 SLIDE 14 SHIFT COMMAND 4 SLIDE 15 SHIFT COMMAND 4 SLIDE 16 SHIFT COMMAND 4 SLIDE 17 SHIFT COMMAND 4 SLIDE 18 Mechanisms of Extinction Conditioned response is not erased after extinction Extinguished fear responses often recover without any additional CS-US retraining This is because extinction learning and memory is closely tied to the context where extinction occurred The original CS-US memory is not context dependent Clinical significance- why would we want to study extinction? 1. To understand how we learn about our environment, to understand how we learn about changes to our environment 2. To understand anxiety a. Uncontrollable fear in non threatening situations b. Exposure therapy is extinction Using extinction in clinical settings Extinction is context specific Learned behaviour reinstates following the brief exposure to the unconditioned stimulus Extinction dissipates over time SHIFT COMMAND 4 SLIDE 22 SHIFT COMMAND 4 SLIDE 23 Adolescents show poor extinction retention Some evidence for sex differences ○ Not always Fluctuating hormones may mediate sex differences If considered by estrous phase: ○ Proestrus (high estradiol) = facilitated extinction ○ Metestrus (low estradiol) = impaired extinction Hormone supplements/blocks can facilitate/impair extinction ○ In females ○ And males ○ Suggests that hormone cause extinction rate ○ SHIFT COMMAND 4 SLIDE 26 In adolescents the effect is opposite Proestrus and Diestrus (high oestrogen) = impaired extinction Summary Extinction is the reduction in Pavlovian conditioned response  ○ E.g Pavlovian conditioned fear Extinction is not the erasure of original learning Extinction is context dependant, original learning can generalise across different contexts Extinction can be used as a therapy for phobias and other anxiety-related disorders Extinction can be used as a therapy for phobias and other anxiety related disorders Extinction varies across development ○ Juveniles: forgetting rather than extinction ○ Adolescents: extinction deficit Extinction is affected by sex/hormones Part 2 Operant Conditioning SHIFT COMMAND 4 SLIDE 31 Mechanisms of Extinction Clinical significance of extinction Reduce unwanted behaviours e.g.... ○ Conduct disorder and other disruptive behaviours ○ Substance use disorder, gambling use disorder Problems with extinction as an intervention ○ What is the reinforcer? ○ Are there more than one reinforcers ○ What is being extinguished ○ Extinction burst ○ Extinguished behaviours can return Clinical significance of extinction and reinstatement Addiction is learning about drugs Extinction can be used to change drug seeking behaviour Reinstatement can be used to model relapse Reinstatement/primed reinstatement Reinforcer itself acts as a “discriminative stimulus”/signal that pellets were available Drug Primed reinstatement Injection of drug provides and interoceptive signal that drug is available Passage of time is enough to cause increase after Conditioned response is not erased after extinction: Reinstatement Renewal Spontaneous recovery  Recovery of responding after extinction occur in operant conditioning as for pavlovian conditioning Stimuli that are presented during reinforced responses from associations with the reinforcer These stimuli can mediate responding after extinction Stress induced reinstatement Stronger for drug rewards, with stress is a lot higher Circulatory of stress induced reinstatement is different from forms of reinstatement Stress shares a common neural substrate with many drugs of addiction Rapid requisition Rat need to learn a new response to get the same reinforcer Resurgence Showing all options that worked in the past Shows an important difference between classical and operant conditioning Classical extinction leaves the CS with two available meanings: 1. CS → US (excitatory conditioning) 2. CS → no US (inhibitory conditioning) Operant extinction leaves the individual with two available options 1. Give up 2. Try other means (having another response available makes extinction faster) Reducing relapse by extinguishing the context drug induced reinstatement, context is important for extinction condition for rate with no cue extinction, they show increase extinction compared to cue extinction Extinguishing behaviour by reducing the reinforcer Run down corridor to get heaps of food pellets Then they still run down but only 6 pellets Extinction when there is less But if there is only 50 pellets the whole time its pretty steady Extinguishing behaviour by providing an alternative reinforcer (social reward) they would rather hang with buddies then get meth Age dependant effects of extinction Adolescents show extinction deficit as with pavlovian conditioning Sex dependant effects of extinction/reinstatement Stress induced reinstatement is stronger in females than in males Summary  Extinction is the reduction in a learned behaviour Extinction is not erasure of the original learning, and original learning can be retrieved ○ Spontaneous recovery (after passage of time) ○ Reinstatement (through non-contingent exposure of the reinforcer) ○ Renewal (through change in context) Also see ○ Resurgence ○ Cue-induced reinstatement ○ Stress-induced reinstatement ○ Rapid reacquisition Extinction can be used to reduce inappropriate fear/anxiety Extinction can be used to eliminate unwanted behaviours Extinction may be used to help reduce craving when treating substance use disorder (but it’s not very effective). In both cases be aware of individual differences (including age and sex) After extinction, the CS or the response has two meanings, context is often used to disambiguate these. Operant responding is supported by many different aspects ○ Associations between the response and the outcome (R-O) ○ Associations between discrete cues and the outcome (S-O) ○ Associations between the discrete cues and the response (S-R) ○ Associations between the context, the outcome and the response (occasion setting) Omitting the outcome is not the only way of changing behaviour ○ Change the magnitude of the reinforcer ○ Add a punishment ○ Provide an alternative reinforcer Week 6 Lecture- Aversive Conditioning: Punishment, Escape and Avoidance Learning Aversive Control: Punishment What is punishment? The weakening of a behaviour through either the: 1. Application of an aversive stimulus (positive punishment) 2. Removal of an appetitive stimulus (negative punishment) Positive punishment ○ Consists of the presentation of an aversive event following a response that leads to a decrease in the future strength of that response Negative punishment ○ Consists of the removal of an appetitive event following a response leads to a decrease in the future strength of that response Punishment ○ Bear in mind that the to be punished behaviour is enjoyable to stop that individual doing the misdemeanour and they won’t want to stop And what we have seen from extinction is that we cannot unlearn a behaviour- we can only suppress or inhibit it; or train an individual to omit to it Two types of negative punishment 1. Time out a. Loss of access to positive reinforcement for a brief time following occurrence of problem behaviour e.g send child to room, likely to be ineffective if time out setting is more reinforcing, or if it is too long of a time out 2. Response cost aka omission training a. Removal of valued reinforcer following occurrence of inappropriate behaviour b. The stimulus that is removed must decrease the frequency of the operant behaviour. The benefit of this is you can adjust the punishment to suit the size of the problem behaviour. Time out → suspension → expulsion c. Taking toys away if they are naughty, swear jar, paying a fine for speeding Negative punishment vs extinction Does the behaviour grow weaker because performing it: ○ no longer leads to something? Extinction ○ Leads to the removal of something that you would otherwise posses? Then negative punishment Similarities ○ Both involve removal of reinforcers ○ Both result in decreasing strength of behaviour Differences ○ Extinction- behaviour that previously produced the reinforcer no longer does and behaviour stops e.g whining no longer produces lollipop ○ Negative punishment- performing the behaviour results in a loss of reinforcer that is already possessed e.g whining results in lollipop being taken away Intrinsic vs extrinsic punishment Intrinsic ○ The behaviour performed is inherently punishing e.g less likely to lift heavy object Extrinsic ○ The event that follows the behaviour is punishment e.g being chastised after posting something inappropriate Primary vs secondary punishers Primary ○ Events that are inherently / innately punishing e.g intense heat, loud noises, hunger pangs Secondary ○ An event that has become punishing because it in the past has been associated with some other punisher Problems with use of punishment 1. Punishment of an inappropriate behaviour does not directly strengthen the occurrence of appropriate behaviour, may result in a suppression of behaviour e.g punishment for swearing may reduce all verbal interactions 2. The person delivering the punishment could become a discriminative stimulus for punishment. The unwanted behaviour is suppressed only when that person is present e.g speed is only curbed when speed cameras are present 3. Might teach individual to avoid that person or context e.g choose alternative route 4. Likely to elicit a strong emotional response, may interfere with any subsequent attempts to teach appropriate behaviour 5. Can elicit an aggressive reaction, directed at punisher or other target 6. Might teach them that punishment is an acceptable means of controlling behaviour, employee gets shouted at and then takes it out on partner 7. Use is often strongly reinforced, punisher may use it more often Benefits Lang and Melamed 1969 used punishment to stop psychogenic vomiting and rumination in a 9 month old boy. Brief and repeated mild shocks that shocked him when he started vomiting, by third session, one or two shocks made him stop vomiting, and then he was dismissed from the hospital 5 days later Effective use of punishment 1. Should be immediate 2. Should be intense enough from the outset to suppress the target behaviour 3. Should consistently follow each occurance of the unwanted behaviour 4. Negative punishment is generally preferable to positive punishment- less harmful side effects 5. More effective when accompanied by an explanation 6. Should be combined with positive reinforcement Conditioned suppression account Does not directly weaken behaviour but proces emotional response that interferes with the occurrence of the behaviour E.g when a rat is shocked, it becomes too frightened to press the lever. Lever response is omitted by fear response Ie replaced behaviour with emotional response or another behaviour that interferes with being able to produce it Effects of punishment are temporary Avoidance account Punishment involved avoidance learning in which the avoidance response consists of any behaviour other than the behaviour that is being punished i.e learn where speed cameras are Another example of the permanency of original learning Summary There are two types of negative punishment ○ Time out ○ Response cost Can be intrinsic or extrinsic Primary or secondary (conditioned) Number of important problems/limitations of punishment To be effective it needs to be administered immediately, consistently and be of sufficient intensity to suppress the behaviour Number of theoretical accounts of punishment ○ Conditioned suppression account ○ Avoidance learning Aversive Control: Escape and Avoidance  Positive punishment motivates escape ○ Escape is getting away from aversive stimulus in progress, results in termination of an aversive stimulus Negative reinforcement reinforces avoidance learning ○ Avoidance behaviour occurs before aversive stimulus preventing the delivery of it Negative contingency between response and aversive stimulus Results in increase in operant conditioning Escape In the presence of aversive stimulus Make response Aversive event terminates Action is negatively reinforced If there is no escape Uncontrollable bad events → perceived lack of control → generalised helpless behaviour If there is escape, and you can anticipate the aversive event before it affects you and evade it? We have avoidance learning Avoidance 1. Active avoidance- making a response to avoid event e.g running away 2. Passive avoidance learning not to make a response to avoid a conflict e.g staying quiet Active avoidance Rat trained in a shuttle box with a hurdle in the middle  Tone for 10 seconds then electric shock If rat avoidance shock when tone comes then the shock is turned off or cancelled This is called signalled avoidance because the experimenter provides a signal to indicate when the shock is imminent Avoidance learning is measured in latency to respond to the signal. This subject avoided shock on the 9th trial and continued to do so thereafter Can learn to use SD to avoid US Passive Avoidance Passive avoidance using a shuttle box procedure It is a form of operant conditioning where the person/animal must abstain from an act or reaction - which will otherwise → have a negative outcome. The animals learn to suppress their normal dark-seeing reflex because their entry into a dark chamber is paired with a foot shock → learn to stay on the bright side of the box Obsessive Compulsive Disorder vs Phobia OCD typically involves an active avoidance response Phobic behaviour typically involves a passive avoidance response Example: ○ A person with OCD will clean frequently or compulsively check things ○ A person with a phobia will avoid object of their fear e.g dogs Avoidance Paradox A response/behaviour is made before the aversive stimulus occurs Behaviour clearly increases, so it is reinforced What is delivered to reinforce it? ○ Therefore not getting something can hardly, in and of itself, qualify as rewarding Bit of a problem for behaviourists who need to be able to specify a stimulus Not getting punished or injured is rewarding only if punishment is expected i.e only if the subject is anxious or fearful, and if this expectation in some ways gets reduced Theories of Avoidance Mowrer's two process theory of avoidance 1. First the subject learns to associate the warning stimulus with the aversive stimulus a. This is a classical conditioning process: b. The warning stimulus of the light in the CS, the aversive stimulus (shock) is the US 2. Now the subject can be negatively reinforced during the warning stimulus; this is the second, operant conditioning process a. Thus the two process theory reduces avoidance learning to escape learning b. The organism learns to escape from the CS and the fear it elicits Avoidance conditioning and phobias Phobia is irrational fear of specific object or situation Disproportionate to the real threat Acquisition → classical conditioning Elevator CS - feeling trapped US - fear UR Elevator CS - fear CR Maintenance - avoidance (negative reinforcement) Elevator SD- avoid elevator RR → reduced fear SR Support for two factor theory Two factor theory predicts that the avoidance responding will be learned only to the extent that the warning signal terminates when a response is made Kamin 1958 trained four groups of rats in a two chamber avoidance apparatus  ○ Terminate signal - avoid shock ○ No termination of signal - avoid shock ○ Terminates signal - does not avoid shock ○ Control Significance amount of avoidance responding occurred in the first group only (response terminates the signal and enables animal to avoid shock As predicted by two factor theory, avoidance responding was poor in the group that was able to avoid shock but could not terminate the signal Further support We know that delaying the onset of reinforcement reduces the effectiveness of reward… so it should be possible to reduce the level of reinforcement by introducing a delay between the avoidance response and the termination of the feared stimulus After the avoidance response, the CS was terminated: ○ Immediately ○ 2.5 seconds after response ○ 5 seconds after response ○ Or 10 seconds after the response As predicted the animals in zero delay avoided successfully in 80% of the trials 10 second delay avoided shock in 10% of trials Effectiveness of CS termination to support avoidance was decreased by increasing delay Results suggest that the source of reinforcement in avoidance conditioning was the reduction of fear generated by the termination of the CS Sidman Free-Operant Avoidance Avoidance can be learned without a warning CS Shocks at random intervals Response gives safe time Extensive training, but rats learn avoidance (errors, high variability across subjects) Bit of a problem for the two factor theory but Have trained dogs using this Every time dog jumped over hurdle, they ensured a shock free period of 30 seconds Once dogs learned avoidance response, confined to half of the shuttle box and given discriminative fear conditioning trials Excitatory can amplify avoidance Inhibitory can reduce avoidance Evidence that the conditioned stimuli have acquired drive properties Supports the two factor theories position that it is the CS that drives the avoidance response Solomon Kamin and Wynne (1953) was against two factor theory Herrnstein and Hineline (1966) One Factor Theory Avoidance is negatively reinforced by the lower rate of aversive stimulation to which it is associated ○ Reduction of aversive stimulation accompanying avoidance maintains avoidance Which theory is correct? ○ Depends Several processes seem to be involved in avoidance learning The fearlessness problem Fear and avoidance are not firmly linked as the theory believes: According to the theory, fear provides the motive to perform the avoidance response Early in experiments a dog would exhibit various signs of fear (whining, urination, shaking) when tone was presented But once the avoidance response is well learned subjects respond without apparent fear Kamin, Brimer, and Black (1963) Rats pressed lever in chamber for food, then in shuttle box associated auditory with shock (excitatory such as tone or light) then put back in operant chamber CS+ from shuttle box presented  Measured suppression of lever press Alternation of behaviour (yo-yo) Every successful avoidance puts CS on extinction With extinction, fear dops so motivation to avoid decreases Resulting in more shocks, strengthening CS again and increasing avoidance response But we don't really see this in dog jumping trial Persistence of Avoidance Cognitivists believe avoidance responding is based not on fear but on subjects expectation that a response will avoid shock During training, warning stimulus is followed by shock, it is assumed that subjects form an expectation that shock will occur when the stimulus is presented When the animal eventually jumps over the barrier to avoid a shock, a new expectation forms (shock does not occur if the response is made So the next time th warning stimulus is presented, the animal recalls both expectations (shock occurs if it doesn't jump but not if it does) and because it prefers not to be shocked, it will perform the response that produces this outcome Cognitive explanation of avoidance learning based on expectations Thus, fear has little role in this theory…it can account for the shortcomings of two factor theory. First re the disappearance of fear during training But the animals continue to jump because it still expects shock to occur if it doesn't jump, and prefers to avoid this outcome Secondly in the difficulty extinguishing avoidance behaviour this also follows directly from a cognitive analysis ○ In the absence of a response shock will occur BUT ○ If the response is made the shock will not occur Early in extinction, the dog holds both of these expectations and therefore responds And when shock does not occur its expectation that responding will not be followed by shock is confirmed and therefore continues to jump Response prevention → extinction Animals don't get a chance to learn what would happen if they did not jump Cognitive theory of avoidance Fear allows it to explain the continuation of avoidance responding in the absence of fear, but not for the same reason it has difficulty explaining evidence that fear does influence avoidance Other factors: response problem Experimenters tried to teach rats to press a bar to avoid press Hundreds of trials are needed to learn to press a bar to avoid shock and yet many never do Because each species have species specific defence reactions, so the particular SSDR that occurs depends on ○ Nature of aversive stimulus ○ Response opportunities provided by the environment SSDR Innate, evolved reactions If familiar escape route is there they can do, otherwise can freeze Predominate in initial stages of avoidance Hierarchy ○ If SSDR works, keep it ○ If not try next Summary Positive punishment/aversive events motivate escape beahviours wich are strengthened via negative reinforcement Involves avoiding the aversive event before it occurs Avoidance learning can take passive or active form Number of theories of avoidance ○ Two factor  Classical and operant conditioning are involved Avoidance is driven by an escape from fear, not prevention of aversive event ○ Cognitive theory of avoidance Responding is based on fear but on subjects expectation that a response will avoid an aversive stimulus Species specific defence responses (SSDR’s ○ Innate response to threat or fear Some avoidance behaviours are more easily learned than others Week 7 Lecture- Behavioural Neuroscience: Genetics Behaviour is the response of an individual to a stimulus (trigger) in the environment Some people (species) respond differently to a similar environments because of: ○ Their genetic make-up (nature) Evolution (e.g human vs bird) Parental traits (colour of skin, height, intelligence, ability to sing) ○ Factors in their environment (nurture/experience) How they are feeling (hungary, tired, hot etc) Previous experiences (or taught a skill or how to behave) Culture Genetics Chromosomes Found in every cell Genes drive the function of our cells 46 chromosomes- 23 pairs, it is unusual to see chromosomes as single structures as they are usually in a replicated stage, there are two of each (sister chromatids) joined together at the centromere Females - XX Males- XY Females always contribute X and males do XY- sex of offspring is alwasy determined by the male Genes Regions of DNA that are units of hereditary (information can be passed from one generation to another) DNA= double strands of nucleotide “base pairs” on chromosomes These are in the nucleus of cells DNA = deoxyribonucleic acid Nucleus  Membrane bound part of a cell which contains all genetic information to promote survival of the cell (organism) Cells are present in all body parts- groups of cells make an organism Each cell contains a nucleus and contains chromosomes made up of genes What and Why Genes Chromosomes are made up of genes Particular sequences of DNA are genes DNA in each gene programmes the manufacturing of different proteins for use in the body Proteins are made with amino acids Amino acids are obtained from our diet and are the building blocks of life 9 Essential Amino Acids 20 used to make proteins 9 of these are essential for us to make proteins which makes us live ○ Histidine ○ Isoleucine ○ Leucine ○ Lysine- beans ○ Methionine ○ Phenylalanine ○ Threonine ○ Tryptophan- can be found in turkey, sesame seeds ○ Valine DNA makes proteins essential for life The sequence of amino acids to form the protein is determined by an intermediate RNA (ribonucleic acid) from DNA DNA is self replicating molecule Each base determines one base of RNA RNA Copy of one strand of the DNA A TRIPLET of bases determines one amino acid Formation of proteins by combining amino acids Body structures or enzymes The Making of Proteins Genetic information gets translated into proteins for use in the body ○ May become biological tissue/cells ○ May become enzymes ○ Enzymes (catalysts) produce chemical reactions so that our bodies function normally - saliva has an enzyme called amylase which helps us to breakdown the sugar that we eat Genetic life cycle: ○ SHIFT COMMAND 4 SLIDE 13  ○ Replication- essential in cell division when making new cells Genes-protein-whole body Group of cells make up an organism Cells are in protein Protein → body parts and enzymes Replication and Division of cells Cells can be directly replicated to produce more cells with the same genetic information (Mitosis) Diploid cells Particular cells divide their genetic information - ready for recombination with another organism (Meiosis) Meiosis 1- separation of chromosome pair Meiosis 2- division of sister chromatids - 4 haploid cells - gametes SHIFT COMMAND 4 SLIDE 16 SHIFT COMMAND 4 SLIDE 17 Homozygous vs Heterozygous Recombination of genes allows for the biological expression of different characteristics (physical or psychological) Our genes on the chromosomes are responsible for making proteins They are also responsible for our characteristics ○ Colour of eyes ○ Colour of skin ○ Height ○ Intelligence ○ Predisposed to run fast If a chromosome contains two identical genes on the chromosome pair we are said to be homozygous for that gene (characteristic) If a chromosome contains two different genes (for the same characteristic) we are said to be heterozygous for that gene E.g homozygous is blue blue eyes Heterozygous is brown blue eyes The characteristic of eye colour is determined by homozygous vs heterozygous genes and if the genes are dominant or recessive Dominant genes only require one gene to be presents on the chromosome to produce that characteristic Recessive genes require both genes to be present on the chromosome to produce that characteristic Eye Colour For eye colour; brown is dominant B and blue is recessive b  Recessive characteristics can only be seen if the gene is homozygous SHIFT COMMAND 4 SLIDE 29 Genetic Alterations Mutations: alterations of deletions of genetic sequence-may be beneficial or not! Failure to divide properly in reproduction, get additional genes e.g Down’s syndrome has 3 chromosomes at chromosome 21 Beneficial Mutations Peppered moth- mutations to black moth Beneficial during sooty industrial revolution Many of our ancestral genes are dormant- can we turn them on again? Newer therapies are aiming at turning genes on or off Epigenetics and gene expression Behaviour can be influenced without mutations (permanent genetic changes) The same genes can be “expressed” or “activated” to different degrees If the chemical environment in a cell changes, histones tighten or release their grip on DNA thus regulating the expression of that gene Heritability The way we behave is predicted by our genetic make-up and moulded by our environment The less an effect of our environment on a particular behaviour shows that the behaviour has high heritability High heritability- behaviour is greatly influenced by genes ○ Organisation and cleanliness skills “just like her Mum/Dad” Low heritability- behaviour is greatly determined by environment ○ Surgeons are trained to be organised and clean Identical twin studies- nature versus nurture Evolution of Behaviour Natural selection: each gene may be kept or eliminated to help survival of a species (turned on or off?) If an animal carries a gene to cause slow running, they are likely to be caught and eaten, limiting their breeding capacity Survival (and reproduction) of the fittest Genetic traits that produce an advantage over others in the same species will be kept (caused through breeding by the survivors) Traits that produce advantages may change over time (with changes in environment) Evolution of behaviour depends on the species Summary  Our behaviour is partly determined by our genetic make-up Genes are sequences of DNA DNA makes up our chromosomes, present in each cell nucleus DNA provides the information required to join amino acids Many amino acids joined together are proteins Proteins are important in our bodies to make new cells or enzymes Genetic information can be replicated in new cells (Mitosis) Genetic information can be divided into gametes (Meiosis) Meiosis enables the recombination (mixing up) of genes through breeding (joining of gametes) Some genes are dominant, some are recessive Behaviours that are determined by genetic make-up have high heritability Genetic traits that help survival of the species are kept by breeding of survivors that have that trait Week 8 Lecture- The Nervous System and Brain Cells The Nervous Systems (Everything is simplified) How do we: feel emotion? Experience reward? See, hear and smell? Co-ordinate movement? Reason or judge? Learn or remember? Feel pain? There are gross distinctions between what different parts of the brain do, and also gross indistinction where they affect each other Understanding basic human physiology Promote homeostasis is keeping all cells happy: Oxygen Nutrients and water Balance H and levels PH=7.4 Temperature 37.2 degrees Rid waste product e.g CO2 H+ = free hydrogen ions All of these factors are important for cell survival Cells can only function in the right environment Too acidic- acidosis, neurons are unable to send messages Too basic- alkalosis, neurons send uncontrolled messages Death results if the PH levels are out of the range from 6.8-8.0 Nervous systems Central (CNS): Cerebrum Cerebellum Brain stem Spinal cord Peripheral (PNS): All nerves outside of the spinal cord Somatic NS Autonomic NS Somatic Nervous System (Voluntary) Spinal cord; skin, muscle, nerve,, ventral roots and dorsal roots Sensing environment is dorsal roots on outer side Ventral and dorsal is different properties They receive signal and go to brain Motor signals to move your arm via the ventral system ALS motor neuron has problems with the ventral roots Peripheral Nerves Cranial nerves (12 pairs) Cervical nerves (8 pairs) Thoracic nerves (12 pairs) Lumbar nerves (5 pairs) frequent for back pain Sacral nerves (5 pairs) Coccygeal nerve (1 pair) Lobes of Cerebrum (cortex) Grey area is cerebellum Yellow below it is brain stem Somatic nervous system (voluntary) Receives sensory information and delivers motor output Received through dorsal roots Motor output is delivered via ventral roots to muscle 6 areas that the nerves join the CNS- cranial, cervical, thoracic, lumbar, sacral, coccygeal Four lobes of the cerebrum (frontal, parietal, occipital, temporal) Brain receives sensory information (of touch) at the somatosensory cortex and produces behaviour by modulating motor output from the primary motor cortex Autonomic nervous system (involuntary) Sympathetic- extends from thoracic and lumbar spine ○ Short preganglionic nerves ○ Long postganglionic nerves Parasympathetic- extends from cranium and sacral spine (craniosacral) ○ Long preganglionic nerves ○ Short preganglionic nerves Both are usually active, but change intensity as the need arises Parallel systems that work in opposition together Increase sympathetic means decreased parasympathetic Sympathetic is four F’s ○ Fight ○ Flight ○ Fright ○ Fuck  Parasympathetic is non emergency ○ Digestion ○ Growth ○ Immune responses ○ Energy storage Pre and post ganglion lengths Red circle and blue circle Pre ganglion on sympathetic is longer than post Post ganglion on parasympathetic is longer than pre Work in opposition on Communication of brain to body Peripheral nervous system ○ Somatic (sensorimotor) ○ Autonomic (parasympathetic, sympathetic) Hormones ○ Hypothalamus & pituitary (many different hormones) ○ Pineal gland (melatonin) ○ Slower and a wider range of effect as it goes through the blood stre Consider: speed and range of effect Many different hormones are released through the pituitary gland and blood flow throughout the blood Keeping the brain happy The brain is highly vascularised (many arteries and veins) which maintain a constant fresh supply of oxygen and nutrients (glucose) to the brain and cells The brain is protected and nourished in several ways ○ The blood brain barrier ○ Cerebrospinal fluid (CSF) ○ Glial cells The blood brain barrier The blood-brain barrier (BBB) is a crucial immunological feature of the human central nervous system (CNS). Composed of many cell types, the BBB is both a structural and functional roadblock to microorganisms, such as bacteria, fungi, viruses or parasites, that may be circulating in the bloodstream Crossing the blood brain barrier Brain capillary endothelial cells have continuous tight junctions  Only highly lipophilic drugs and small uncharged molecules can cross from blood capillaries to cerebrospinal fluid (CSF) by diffusion Important nutrients- amino acids and glucose are actively transported across to CSF by proteins in the capillary membrane called transporters. Transportation across the membrane requires energy use Cerebrospinal fluid and ventricles Ventricles are big holes in your brain Order of operations of generating cerebral spinal fluid 1. Created at choroid plexus of lateral ventricle 2. Circulates to 2 3. Then down the cerebellum 4. Then around brain at the top Ventricles and CFS CFS is made by choroid plexus in lateral and third ventricles The ventricles and subarachnoid space circulate cerebrospinal fluid through and around the brain to provide nourishment and protection (cushioning) of neural tissue There are four ventricles (lateral, third, fourth and cerebral aqueduct) Once CFS has circulated around the brain it is reabsorbed by the arachnoid villi (of subarachnoid space) into venous blood (return to heart) The cranial and spinal meninges also protect and nourish these ares: dura mater, arachnoid mater and pia mater Part 2 Brain Cells Neuron Network The human brain has approximately 100 billion neurons and even more glial cells Huge network for communication and complex processing White vs Grey matter Grey matter is neuronal cell bodies White matter is the axon before the cell body covered in the fatty sheath Discovering brain cells was not easy Neurons Glial cells- astrocytes and oligodendrocytes Ependymel cells- line with the CSF filled ventricles (neurogenesis) Microglia- remove dead or degenerating neurons or glia (phagocytosis) Neurons Most have 4 main parts: Soma- middle, nucleus Dendrites- spiny bits around receive signals Axon- if fires, passes down axon Presynaptic terminals- to here, and then they communicate with other cells Typical Neuron Axon + Dendrites = Neurites Classification of Neuron Type Neurons are classified by The number of neurites from cell body Their dendrites- how many and if they have spines or not Their axon length ○ Golgi type 1- long “internuncial” ○ Golgi type 2- small “interneurons” The neurotransmitter used by the neuron Neuronal connections- primary sensory neurons or motor neurons Classification by neurites Cells can be unipolar, bipolar or multipolar- defined by the number of neurites connected to the soma Pyramidal cells: ○ An apical dendrite extends from the apex of the pyramidal cell soma ○ Basilar dendrites are connected to the base of the soma (not the axon) Classification by Dendrites Neurons are classified by the shape and kind of dendrites Stellate (starshaped A or D on slide 6) vs pyramidal (triangular) Has spines (spinous) or doesn’t have spines (aspinous) Dendritic spines are involved in learning and memory Dendritic trees constantly change (grow or recede)- aids in neuroadaptation Dendrites are sources of information for the neuron- the more dendrites the more info the neuron receives Grey area is a amyloid plaque, the cell that's near the plaque has fewer dendrites than ones further away More plaques has memory and learning problems, less dendrites is associated with memory loss and cognitive deficits in Alzheimer’s disease Afferent vs Efferent Neurons Afferent is to the connection Efferent is from the connection  They are not touching, they are very close together ad the arriving neurotransmitter releases its energy into the space and then the exiting can pick it up Summary Cortex is composed of white (myelinated axons) and grey matter (neuron cell bodies) Brain and cells consist of neurons, glial cells (glia), ependymal cells and microlglia There are 4