Operant Conditioning 1 Learning
1.  Classical conditioning
• 
Associate two stimuli • 
Associate a response and its consequence, and repeat acts followed by rewards and avoid acts followed by punishments
2.  Operant conditioning
3.  Observational learning
• 
We learn from others’ experiences and examples (e.g., a kid modeling his father dealing with a problem) 2 Operant & Classical Conditioning
• 
Classical conditioning forms associations between stimuli (CS and US).
Operant conditioning forms an association between behaviors and the resulting events.
Operant behavior, a behavior that operates on the environment, producing rewarding or punishing stimuli.
3 Operant Conditioning
Behaviors are associated with their consequences
  Behavior is   strengthened if followed by reinforcement   diminished if followed by punishment
4 Operant Conditioning
Reinforcement: Any event that strengthens the behavior it follows.
Punishment: An aversive event that decreases the behavior it follows.
5 5 6 Early work: Thorndike Law of Effect •  On the basis of his work, Thorndike formulated the law of effect •  This law states that “Rewarded behavior is likely to occur again.”
Skinner’s Experiments
Skinner’s experiments extend Thorndike’s thinking, especially his law of effect.
(Thorndike) Law of effect:
Rewarded behavior is likely to occur again.
9 Processes of Operant Conditioning
 
 
 
 
Acquisition   A response is followed by a reinforcer.
Extinction
  A response is no longer followed by a reinforcer.
Stimulus generalization
  Stimuli that are similar to the original stimulus are more likely to trigger a response.
  Pass at different shades of “green” traffic light.
Stimulus discrimination
  The tendency of a response to occur in the presence of one stimulus but not another.
  Stop at “red” traffic light and pass at green. 10 10 Operant Chamber
Using Thorndike'ʹs “law of effect” as a starting point, Skinner developed the operant chamber, or Skinner box:
It comes with a bar or key that an animal manipulates to obtain a reinforcer like food or water. The bar or key is connected to devices that record the animal’s response.
11 Operant Chamber
12 Shaping
Shaping is the operant conditioning procedure in which reinforcers guide behavior towards the desired target behavior through successive approximations.
To teach complex behaviors may need to reinforce successive approximations of a desired response.
Example: training animals, geYing children to make their beds
DemonstraFon 13 “SupersiFon” in the pigeon Interview DemonstraFon Types of Reinforcers
Reinforcement: Any event that strengthens the behavior it follows. 15 Examples for Types of Reinforcers
Positive Reinforcer:
•  Watching your favorite television show after finishing your assignment: Behavior increased (doing homework) •  Giving bonus points for aYendance: students start to come more often
Negative Reinforcer:
•  Pressing the snooze buYon of the alarm clock to make it stop
•  If a child cleans his room to avoid arguments from his parents
16 Punishment
An aversive event that decreases the behavior it follows.
Powerful controller of unwanted behavior.
Examples:
•  Keeping your dog in the doghouse for eating your shoes : eating shoes decreases
•  Giving zero to a student who cheats in the exam: cheating decreases
17 17 Punishment
Positive Punishment: Administer an aversive stimulus (e.g., spanking, receiving a parking ticket).
Negative Punishment: Withdraw a desirable stimulus (e.g., time-­‐‑out from privileges)
18 Operant Conditioning
Reinforcement
Positive
Pls Stimulus Presented
Negative
Unpl Stimulus Punishment
Removed
Behavior
Decreases
Behavior...
Behavior...
Increases
Increases
19 19 Operant Conditioning
20 Primary & Secondary Reinforcers
1.  Primary Reinforcer: An innately reinforcing stimulus like food or drink. ( i.e., satisfies a biological need)
2.  Conditioned Reinforcer (Secondary Reinforcer): A learned reinforcer that gets its reinforcing power through association with the primary reinforcer (what is the mechanism?).
e.g.: money, good grades, employee of the month award
21 21 Immediate & Delayed Reinforcers
1.  Immediate Reinforcer: A reinforcer that occurs instantly after a behavior. A rat gets a food pellet for a bar press.
2.  Delayed Reinforcer: A reinforcer that is delayed in time for a certain behavior. A paycheck that comes at the end of a week.
22 Reinforcement Schedules
1.  Continuous Reinforcement: Reinforces the desired response each time it occurs.
•  Learning occurs rapidly
•  Faster acquisition, faster extinction
2.  Partial Reinforcement: Reinforces a response only part of the time.
•  Results in slower acquisition
•  Shows greater resistance to extinction.
23 Reinforcement Schedules
Partial Reinforcement (cont.): Reinforces a response only part of the time.
Can be rewarded based on frequency of behavior (ratio) or time (interval):
• 
• 
Ratio schedules: Reinforcement is based on the number/frequency of behaviors required.
Interval schedules: Reinforcement is based on the passage of time.
24 Reinforcement Schedules
•  There are 4 types of reinforcement schedules:
–  Fixed ratio
–  Variable ratio
–  Fixed interval
–  Variable interval
•  Each of these schedules will produce different response paYerns 25 Ratio Schedules
1.  Fixed-­‐‑ratio schedule (FR):
Reinforces a response only after a specified number of responses. E.g., piecework pay, frequent flyer programs
  faster you respond the more rewards you get
  very high rate of responding   resting reduces rewards
26 Ratio Schedules
2.  Variable-­‐‑ratio schedule (VR):
Reinforces a response after an unpredictable number of responses.
This is hard to extinguish because of the unpredictability.
E.g., behaviors like gambling (slot machines)
27 Interval Schedules
1.  Fixed-­‐‑interval schedule (FI):
Reinforces a response only after a specified time has elapsed.
Response occurs more frequently as the anticipated time for reward draws near
E.g., preparing for an exam only when the exam draws close.
28 Interval Schedules
2.  Variable-­‐‑interval schedule (VI): Reinforces a response at unpredictable time intervals, which produces slow, steady responses. Produces slow steady responding because there is no way of knowing when the wait will be over.
E.g., pop quiz, email checking
29 30 Schedules of Reinforcement
31 Extending Skinner’s Understanding
Cognition & Operant Conditioning
Skinner believed in inner thought processes and biological underpinnings, but many psychologists criticize him for discounting them.
32 Cognitive Map
• Mental representation of the layout of one’s environment.
• Example: after exploring a maze, rats act as if they have learned a cognitive map of it even when not rewarded along the way
• Learned on its own.
Demo 33 Latent Learning Control Group (Group I): 1 trial per day Finds food at the end of the maze at the end of each trial Experimental Groups (Group II): Never reinforced (Group III): No food found in the first 10 days. Food present from day 11 and on. Latent Learning
Such cognitive maps are based on latent learning, which becomes apparent only when an incentive is given. So there is more to learning than associating a response with a consequence. 35 Latent Learning 36 SpaFal OrientaFon Experiments SpaFal OrientaFon Experiments •  Rats were trained in this maze for 4 nights, 3 trials per night •  Then, the apparatus was manipulated SpaFal OrientaFon Experiments Intrinsic Motivation
Intrinsic Motivation: The desire to perform a behavior for its own sake.
Extrinsic Motivation: The desire to perform a behavior due to promised rewards or threats of punishments.
40 Intrinsic Motivation
Rewards may not all be too helpful at times.
Unnecessary rewards can be costly:
Promising people a reward for a task they already enjoy can backfire.
41 Turning Play Into Work
•  When preschoolers were promised a prize for drawing with felt-­‐‑
tip (colorful, bright) pens, the behavior increased.
•  After they got the prizes, they spent less time with pens than before the study began.
42 Biological Predispositions
Biological constraints predispose organisms to learn associations that are naturally adaptive.
e.g. You can easily teach pigeons to flap their wins to avoid being shocked, and to peck to obtain food, because fleeing with their wings and eating with their beaks are natural pigeon behaviors.
43 Biological Predispositions
•  “However, a second instance involving a raccoon does not fit so neatly into this paradigm. The response concerned the manipulaFon of money by the raccoon (who has "hands" rather similar to those of the primates). The conFngency for reinforcement was picking up the coins and deposiFng them in a 5-­‐inch metal box.” •  “Raccoons condiFon readily, have good appeFtes, and this one was quite tame and an eager subject. We anFcipated no trouble. CondiFoning him to pick up the first coin was simple. We started out by reinforcing him for picking up a single coin. Then the metal container was introduced, with the requirement that he drop the coin into the container. Here we ran into the first bit of difficulty: he seemed to have a great deal of trouble leang go of the coin. He would rub it up against the inside of the container, pull it back out, and clutch it firmly for several seconds. However, he would finally turn it loose and receive his food reinforcement. Then the final conFngency: we put him on a raFo of 2, requiring that he pick up both coins and put them in the container.” •  “Now the raccoon really had problems (and so did we). Not only could he not let go of the coins, but he spent seconds, even minutes, rubbing them together (in a most miserly fashion), and dipping them into the container. He carried on this behavior to such an extent that the pracFcal applicaFon we had in mind -­‐ a display featuring a raccoon puang money in a piggy bank -­‐ simply was not feasible. The rubbing behavior became worse and worse as Fme went on, in spite of non-­‐reinforcement.”
44 Applications of Operant Conditioning
Reinforcers affect productivity. Many companies now allow employees to share profits and risks, and participate in company ownership.
At work
45 Applications of Operant Conditioning
At Home
In children, reinforcing good behavior increases the occurrence of these behaviors.
Ignoring unwanted behavior decreases their occurrence.
46 Operant vs. Classical Conditioning
47