Instrumental Conditioning: Motivational Mechanisms

• Lecture 6 is about Instrumental Conditioning: Motivational Mechanism
• The two major mechanisms are Associative structure of instrumental conditioning and behavioral regulation

Associative structure of instrumental conditioning (molecular)

• Thorndike's work is similar to the tradition of Pavlov
• Focus on individual responses and their specific stimuli antecedents and outcomes

Behavioral regulation (molar)

• Skinner is associated with behavioral regulation
• Focus is on the long-term goal and how behavior is regulated in the face of limitations or restrictions created by an instrumental conditioning procedure
• The mechanisms that motivate and direct instrumental responses.

The associative structure of instrumental conditioning

• The "building blocks" are described by this section
• Uses as S-R-O system, focuses on individual responses
• The three-term contingency includes
• S (Stimulus context)
• R (Instrumental response)
• O (Response outcome/Reinforcer)
• Instrumental procedures allow for the development of several different associations which are Binary and Hierarchical

Binary Association

• Includes S-R, S-O and R-O

Hierarchical Association

• Includes S(R-O)
• Instrumental response (R) occurs in presence of distinctive stimuli (S) and results in delivery of reinforce outcome (O).

Thorndike's S-R association

• Law of effect and Motivation are key here
• Reinforcer "stamps in" (strengthens) the S-R association
• Behaviors become habitual, which is an automatic reaction
• Context can even induce drug seeking
• Instrumental conditioning doesn't always involve learning about the reinforcer (O) or the relationship between R-O,

Thorndike believed

• Animals do not always "know" why they were running the maze
• Animals do not always “expect” reward
• Behaviors were often robotic, "stamped in" by the reinforcer (O)
• This view was really hit hard by the cognitive revolution
• S-R explanations are "habitual nature" which are performed without mediation of goal
• Drug addiction, compulsive eating, gambling, or infidelity are examples
• Context (S) can induce drug seeking (R)
• Response is automatic without considering the consequence

Two-Process Theory

• States there are two distinct kinds of learning: Classical and Instrumental conditioning
• Classical and instrumental are related in a way
• Classical conditioning focuses on CS-US association
• Instrumental Conditioning S(context)-O(outcome) association leads to an emotional state Motivates R

S-0 association

• Two-process Theory states that an emotional state that motivates R
• Chamber (S) -Food (O) -> Hope (Emotion) -> Enhances opening door (R)
• Maze (S) -Shock (O) -> Fear (Emotion) -> Inhibits opening door (R)

Testing the Role of Emotional States: Pavlovian-Instrumental Transfer Test

• Phase 1 involves Instrumental Conditioning (lever press -> food)
• Phase 2 involves Pavlovian Conditioning (Tone + Food)
• Test involves Presenting Pavlovian CS (Tone) during performance of instrumental responses
• Two-Process Theory can be tested- when should animals respond faster?
• The presence of the tone (CS) intensifies Instrumental responding in the test phase
• Positive emotion elicited by the tone (CS) for food increases the rate of lever pressing for food because S-O association further motivates responding
• Thorndike explanation of instrumental responding & Two-Process theories ignore relations
• The R-O association is important, because "common sense" says that animals do associate outcomes with particular responses
• An example is : Push button on remote > expect visual reward and Open door on fridge > expect food reward
• Evidence: selective response suppression by devaluating reinforcers

Experiment 7.1 – Devaluation of Reinforcers

• A selective suppression of responses by devaluating the reinforcers was demonstrated
• Procedure: in Phase 1 Rats push a rod, and Left > Food pellets, Right > Sucrose water,
• In Phase 2: the food or sucrose is Devalued by an injection of poison (taste aversion)
• In Phase 3: Test rats' responses of pushing rod after devaluing
• The Stimulus (rod) is the same across the experimental conditions
• Devaluation should not alter the responses because S-R association does not involve reinforcers(O)
• If devaluation of one of the reinforcers had altered the properties of S, that should have changed the two responses equally
• "Instead, devaluation of a reinforcer selectively suppressed the particular response that had been trained with that reinforcer"
• R-O association is not enough to fully explain it all
• Skinner (1938) suggested an S(R-O) relation is the most complete

Example of S(R-O) Relation

• Watch a perfume advertisement (S) > {Buy perfume and wear it (R) -> Get a kiss from boyfriend (O)}
• The view of instrumental behavior is quite different from the associative account
• How do instrumental contingencies put limitations on an organisms' activities and cause redistribution of those activities?
• How does reinforcement of responding in the presence of a particular stimulus affect behavior?
• There is a big change in view of reinforcers: no longer stimuli but actually preferred behaviors, like Eating, drinking, licking, chewing, swallowing,

The Premack Principle

• States that there exists of two responses of different probabilities
• H: high likelihood (eating dessert) and L: low likelihood (eating vegetables)
• Of any two responses, the more probable response will reinforce the less probable one.
• The opportunity to perform H after L will result in reinforcement of L but not when L comes after H
• States to follow "grandma's law" or "eat your veggies before dessert!"
• A baseline measurement of both activities should be taken
• An illustration of the principle - First graders eat candy or play pinball to demonstrate, that more candy results in kids playing games (like pinball)
• Demonstrates that running (high probability), like sucrose solution, can reinforce instrumental behaviors in rats

Experiment 7.2 – Premack Principle

• Lever pressing (R) in done in rats, reinforced by either Wheel-running or with Different concentrations of sucrose

Premack principle in dog training

• Premack exercise requires to get the prize, the dog must pass it by & do something else first.

Applying Premack principle to increase your productivity

• Money, Time off from work, Lunch time, Working next to Wanda, Supervisor Praise = Most desired
• Getting printing plates, Throwing out oily rags, Typesetting, Cleaning the press = Least desired

Premack principle: Homme et al

• (1963) shows a case study using unruly 3 year-olds
• Used Ignoring Teacher, Screaming, and Pushing Furniture (High Probability), and Sitting Quietly (Low Probability)
• "The Reward for sitting quietly was with 3 min of running around screaming which resulted in the behavior of Sitting quietly increased"

Premack Principle: Applications

• Used with children with Autism who show some problematic (High Probability) High probability behaviors such as Delayed Echolalia, and Perseverative Behavior
• Low probability might include Adding up coins, or Judging objects: same or different

Experiment 7.3 – Reinforcing Behaviors Among Children with Autism

• Explored the effect of Reinforcement on task performance and Reinforce by giving Delayed echolalia/perseverative behaviors
• The results show is nicely demonstrated by theory and is also strictly empirical
• Contains no hypothetical constructs, no references to unobservables like hunger and references to pleasurable vs. nonpleasurable things

Restriction of the Reinforcer

• Is the critical factor and is the Defining element in instrumental conditioning for
• "All procedures require withholding the reinforcer until the specified instrumental response has been performed"
• Holds a behavior at below baseline level, and makes access to it will now become a reinforcer, which is described by Response-Deprivation Hypothesis

Response-Deprivation Hypothesis: Premack revised

• States that Depriving subjects of a low-probability response can make access to that response an effective reinforcer
• Johnson's et al., 2003 found that students who have mental retardation who completed the low probability responses: filing cards while tracing letters, means access to tracing is restricted below baseline levels

Behavioral Regulation

• Is about the extent to which instrumental contingencies disrupt behavioral stability, and force the subject away from preferred or optimal distributions of responses
• The subject must balance activities to redistribute responses to maximize bliss

The Behavioral Bliss Point

• Is where if we can, we distribute our behavior in a way that is optimal
• The bliss point can be determined like Premack did
• Time spent engaging in each activity
• Watching TV
• Studying

The Minimum-Deviation Model

• Shows that the student's preferred time to watch TV (60m) than studying (15m) is disrupted
• The student is only allowed to watch TV for the same amount of time that they study
• The student can no longer achieve the bliss point and must redistribute their behavior

The student must redistribute

• He must reach a compromise, using a method that is called a "Minimum-deviation model"
• Means that the rate of one response is brought as close to its preferred level as possible without moving the other response too far away from its preferred level
• Reorganization of behavior occurs in a way that minimizes the total deviation of the two responses from the bliss point
• The filled circle on Fig 7.8 represents 37.5 minutes of being spent on both, where 22.5 more minutes of studying and 22.5 less minutes of TV are done
• Problems with the bliss point approach have to do with external validity, and that it is a Molar model, not causal like by associative approach

Contributions of Behavioral Regulation

• Show that one must move away from thinking of reinforcers as special stimuli or special responses
• Reschedule behavior distribution
• Distinguish instrumental and reinforcer responses by role they play in a given procedure
• Assume benefit-maximizing
• Consider broader response options available

Main Differences

• The main difference between the approaches:
• Associative: uses more molecular/ lower level, uses S-R, S-O, R-O and all their connections
• Behavioral Regulation: is a molar level (higher), better contact with the ecology of organisms
• Associative is process and casualty oriented whereas Behavioral Regulation is long-term and functional
• Associative approach has Extensibility in that its capability to be integrated with neuroscience research
• Behavioral Regulation has Applicability in therapy and development on clinical perspectives

Lecture on instrumental conditioning's motivational mechanisms, focusing on the associative structure of instrumental conditioning (Thorndike's S-R-O system) and behavioral regulation (Skinner's long-term goal approach). It explores how stimuli, responses, and outcomes create motivation. The lecture dives into the interaction of these elements in shaping behavior.