L4: Operant Conditioning (PSYC2050)

• Operant conditioning principles were discussed in PSYC2050 lecture 4

Classical vs Operant Conditioning

• Ivan Pavlov is associated with classical conditioning
• Classical conditioning involves learning through association
• Classical conditioning was covered in lectures 2 and 3
• B.F. Skinner is associated with operant conditioning
• Operant conditioning involves learning through reinforcement
• Operant conditioning was covered in lectures 4 and 5

Topics Covered

• Shaping (successive approximations)
• Classification of consequences
• Schedules of reinforcement
• Reward variables
• Stimulus control

Operant vs Pavlovian Conditioning

• Pavlovian conditioning relies on forming reflexive associations between stimuli, resulting in involuntary responses
• Operant (or instrumental) conditioning relies on consequences of past actions to influence future behavior, increasing or decreasing voluntary behaviors

Operant Conditioning Principle

• Consequences lead to changes in voluntary behaviors
• Behaviors resulting in rewards are repeated or become more frequent
• Behaviors resulting in punishment are avoided or become less frequent

History - Thorndike

• Edward Thorndike used puzzle boxes with cats
• Cats could escape by pulling a string, stepping on a platform, and turning a latch
• With experience, cats got quicker at escaping

The Law of Effect

• Responses accompanied by satisfaction are more firmly connected with the situation, making them more likely to recur
• The tendency to perform an action increases if rewarded and weakens if not

Skinner Box

• A Skinner box is an apparatus to study operant conditioning

Teaching New Behavior

• One option is to wait for the desired behavior to occur
• Another option is to reinforce any behavior that could lead to the desired behavior, known as shaping
• Shaping involves selective reinforcement of behavior resembling the desired target behavior

Real World Learning

• Real world reinforcement and animal behavioral adaptations to environmental feedback exists even without an experimenter/trainer.

Superstitious Behavior

• Skinner randomly rewarded pigeons every 15 seconds and found they repeatedly performed distinct behaviors between food presentations
• Display of these behaviors was self-perpetuating and can be considered superstitious behavior
• Superstitious behavior reflects a belief that actions cause rewards, even without causation
• Random reinforcement can shape behavior

Superstitious Behavior Examples

• Superstitious behavior exists beyond pigeons
• Athletes' warm-up rituals and lucky clothes are examples of superstitious behavior
• People try to find links between behaviors and outcomes, such as repeatedly pushing pedestrian crossing buttons

Techniques to teach new behavior

• Animal training can make use of shaping, baiting, mimics and sculpting.
• Shaping (Scan & Capture)
  • Rewarding successive approximations of a behavior until the full behavior is achieved.
  • Example: Teaching a dog to roll over by first rewarding a head turn, then a partial roll, then a full roll.
• Baiting -Using a desirable stimulus (a reward) to lure the subject into performing the behavior. -Example: Holding a treat above a dog’s head to make it sit.
• Mimicking -The subject learns by observing and copying the behavior of another individual. -Example: A child learns to clap by watching their parent do it first.
• Sculpting -Physically guiding the subject through the desired movement until they learn to do it themselves. -Example: A swimming instructor moves a child’s arms in a breaststroke motion until they can replicate it on their own. Or when trying to get a dog to come to you at your command, pull their leash in your direction, then give them reinforcer
• Instruction (Language) -Teaching through verbal or written guidance rather than trial-and-error learning. -Example: A teacher explains long division step by step before having students practice on their own.

Teaching New Behavior: Chaining

• Many behaviors are made up of smaller behaviors
• Acquiring a behavior is easier if done in bits and pieces
• Chaining can be done forwards or backwards
• It’s often best to start with the last behavior in the chain, and work backwards, called backward chaining

Reinforcers and Punishers

• Consequences of actions determine the likelihood of that behavior happening again
• A reinforcer increases behavior
• A punisher decreases behavior

Types of Reinforcers and Punishers

• Positive reinforcement involves adding something
• Negative reinforcement involves subtracting something
• Positive punishment involves adding something
• Negative punishment involves subtracting something
• Whether something is "good" or "bad" does not make it positive/negative

Operant Conditioning Contingency Square

• Reinforcement increases the frequency of desirable behavior
• Positive reinforcement adds something and isn’t necessarily good
• Negative reinforcement takes away something and isn’t necessarily bad
• Punishment decreases the frequency of undesirable behavior

Positive Reinforcement Examples

• Providing a treat when a dog sits
• Finishing homework and getting to have ice cream
• Gold stars for good behavior

Bridging

• Teaching the association between a stimulus can be delivered immediately and subsequent reward.
• The stimulus becomes the "conditioned reinforcer" which bridges the time between behavior and primary reinforcement.
• Bridge examples: short phoneme, whistle, clicker

Positive Punishment Examples

• Administering a shock for bad behavior
• Anti-barking collars
• Getting told off following undesirable behavior

Negative Reinforcement Examples

• Removing discomfort increases behaviour
• A student no longer has to do homework because of good marks

Negative Punishment Examples

• Removing something decreases a behaviour
• Losing license for bad driving
• Being put in time out

Schedules of Reinforcement

• Continuous reinforcement (CRF) occurs after each response.
• Partial reinforcement (PRF) occurs only after some responses
• The schedule can be based on ratio (instances of behavior) or interval (time of behavior)

Partial Reinforcement Schedules

• Fixed Ratio (FR): reinforcement after every nth response (e.g., newspaper delivery)
• Variable Ratio (VR): reinforcement after an average of every nth response (e.g., gambling)
• Fixed Interval (FI): reinforcement for the first behavior after N seconds (e.g., waiting for a bus)
• Variable Interval (VI): reinforcement for the first behavior after an average of n seconds (e.g., checking email)

Ratio vs Interval Schedules

• Ratio schedules are generally most efficient
• VR schedule is most resistant to extinction because it teaches persistence

Schedules of Punishment

• Continuous punishmentis the most effective

Effectiveness of Punishment vs. Reinforcement

• Reinforcement is more effective than punishment

Drawbacks of Punishment

• Punishment isn't as permanent as reinforcement
• Punishment reduces trust and increases aggression

Suggestions for Effective Punishment

• Ensure there is no escape.
• The punishment should be as intense as possible, within limits
• Use a continuous schedule
• Administer punishment immediately
• The punishment should be over a short period of time
• Ensure there is no subsequent reinforcement
• Reinforce incompatible, appropriate behavior concurrently.
• Watch for side effects (changes in other behaviours, aggression, fear, modelling of violence and learned helplessness)

Additional punishment information

• No Escape --The child should not be able to escape the punishment while it is being applied, which ensures the child learns that misbehaving leads to negative consequences and that they can't avoid it by running away or hiding. --If a child is told to sit quietly as a consequence for misbehaving, they should remain in the designated area (e.g., corner or timeout spot) without being allowed to leave.

• As Intense as Possible (Within Limits)

• --The punishment should be strong enough to deter the undesirable behavior but not so intense that it causes harm or excessive distress. --A firm but calm verbal warning like "That's not acceptable, you will have to stop playing video games for 10 minutes" can be more effective than a long-term ban, which could cause unnecessary emotional distress.

• Continuous Schedule --Apply punishment every time the undesirable behavior occurs, which helps the child connect the behavior with its consequences. --If a child interrupts while you're speaking, you immediately stop talking and give them a consequence each time, so they understand that interrupting always leads to a timeout.

• No Delay --The punishment should follow immediately after the misbehavior, as the closer the punishment follows the action, the more likely the child will associate the behavior with the consequence. --If a child hits another child, immediately take them aside for a timeout, rather than waiting until later in the day, making the punishment effective.

• Over a Short Period of Time --Punishment should be brief, not prolonged, as long periods of punishment may lead to frustration or resentment and won't allow for immediate learning or behavior modification. --If a child is sent to their room for a timeout, it should last no longer than five to ten minutes, enough to calm down but not so long that it becomes overly punitive.

• No Subsequent Reinforcement --Avoid reinforcing the child after punishment, meaning there should be no rewards or comfort given during or immediately after the punishment, as this could counteract the lesson being taught. --If a child is sent to their room for a timeout, don’t give them any toys or comfort items immediately afterward to "make them feel better," which reduces effectiveness of the punishment.

• Reinforce Incompatible, Appropriate Behavior Concurrently --While applying punishment, you should also reinforce positive behaviors, which encourages the child to engage in desirable behaviors, constructive way to gain positive attention.

• -If a child is punished for not doing their homework but also praised when they do their homework correctly, they will be more likely to engage in the appropriate behavior in the future.

• Watch for Side Effects --Monitor the child’s emotional and behavioral responses to punishment to ensure there are no unintended consequences, Common side effects of punishment include aggression, fear, or learned helplessness. --Ensure there are no unintended consequences, Aggression: If punishment triggers anger or frustration, the child might act out inappropriately later. --Fear: Punishment could create fear of authority figures or situations, leading to avoidance or anxiety. --Modeling of violence: If punishments involve physical force or threats, the child might learn to use aggression to solve problems. --Learned helplessness: If punishments are too frequent or harsh, the child might give up trying to behave appropriately because they feel powerless to change the situation. -- If a child becomes overly anxious after being punished for a minor infraction,you may need to adjust the punishment and use more constructive methods of teaching appropriate behavior.

• Shaping introduces a new behaviour into an animal’s repertoire by reinforcing each time the animal comes closer to performing the desired behaviour

Reward Variables

• Conditioning is affected not only by the schedule, but also by drive, size, and delay

Reward Variables: Drive, Size and Delay

• Reinforcement depends on how much the organism wants the reinforcer
• Reinforcers that trigger an organism's high desire level is more effective
• In operant conditioning, size matters
• Animals in a Skinner box learn faster if they get more food pellets, but there is diminished returns
• Reinforcers that are large/desired have faster acquisition and extinction
• Reinforcement reduces the effect of delay

The Three Term Contingency

• The discriminative stimulus sets the occasion
• The operant response is the behavior
• The outcome (reinforcer/punisher) is the consequence
• Skinner argued that these three terms are the basis of operant conditioning

Stimulus Control

• In the three-term contingency, a discriminative stimulus signals when a particular behavior will be reinforced/punished
• Learning to discriminate the stimulus is key to operant conditioning
• Stimuli become signals if they are predictive of a consequence

Stimulus Generalization

• When a response is reinforced in the presence of one stimulus, there is a general tendency to respond in the presence of new stimuli that have similar properties or have been associated with the stimulus
• Stimulus generalization involves a loose degree of stimulus control

Stimulus Discrimination

• Stimulus Discrimination refers to the degree to which different stimuli set the occasion for particular responses
• It involves precise degree of stimulus control
• Taught by using discrimination training procedures such as differential reinforcement

Stimulus Control examples

• Occurs when your behaviour comes to be under the control of the stimulus
• The behaviour happens when the stimulus is present and doesn’t happen when the stimulus is absent
• Happens during Much of our everyday behaviour

Learning Objectives

• Understand the basis of operant conditioning
• Understand the classification of consequences
• Understand the link between schedules of reinforcement and different rates of learning
• Understand how punishment can be done effectively
• Understand the various reward variables that affect reinforcement
• Understand the mechanism of stimulus control