November 26, 2010

Intermittent Reinforcement

Some people (ahem) check their personal email an absurd number of times per day. Sitting in front of a computer all day, it’s hard to resist checking for new messages, even if you just checked five minutes ago. In fact, it’s almost impossible. Why?
I think I know, and I think I learned it in Intro to Psych in 1999.
In the 1950s, behavioural psychologists experimented with learning by rewarding caged rats with a food pellet every time the rats pressed a lever. Although the rats initially only pressed the lever randomly or by mistake, they quickly learned the relationship between behaviour and reward and responded with frantic lever-pressing. In psych terms, the food reinforced the lever-pressing behaviour, that is, made it more likely to be repeated. Real-life examples of this kind of conditioning include rewarding your child with a new toy when he makes his bed or your company rewarding every two years of service with a pay increase. Toys and pay raises increase bed-making and company loyalty, respectively.
What does this have to do with email checking? Well, once the rats had clearly learned the relationship between lever-pressing and food, researchers started experimenting with the timing and probability of the reward. They wanted to know what would happen if they gave a rat a food pellet every third time it pressed the lever rather than every single time. Or if they provided food every 60 seconds no matter how many times the rat pressed the lever in the past minute. Or if the reward was completely random, that is, independent of timing and frequency of the behaviour. These variations are called reinforcement schedules. To their surprise, the researchers found that the most successful reinforcement schedule was intermittent reinforcement, that is, random and inconsistent reinforcement. This finding has since been widely replicated, in animals and humans, across situations and types of reward. 

Let’s go back to email checking now. You arrive at work in the morning and check your personal email first thing. You have a bunch of new messages. When you check again 15 minutes later, you have two more new messages. The next three times you check, there’s nothing. After lunch, still nothing. But an hour later, in mid-afternoon, you sign in again and bingo–you’re rewarded with 3 new messages! Yessss!! You feel pleased and validated because all your hard email-checking work paid off. You read your messages, respond or delete, and return to work. But 15 minutes later you have the urge to check your email again. You're in the clutches of email's inherently intermittent reinforcement!
Question: Is it disturbing or reassuring to realize that your personal behaviour is governed by basic principles of learning theory that apply to all people? And, um, all rats.


  1. Bingo! A comment on your post!

  2. It occurs to me that if you only update your blog intermittently, you will have us all in your thrall!

  3. Thanks for the justification. I feel justified now in checking my email 25,000 times today.

  4. Do rats appear to have a related fear of missing a pellet, like something terrible will happen if a pellet is available and they don't press the lever asap?

  5. Adore this post. And the logic behind it. I, however, want there to be ZERO EMAILS IN MY INBOX! Or more accurately, 1,000 'please sign me up for Fetching emails', 2 'I love you from my boyfriend' emails, and zero 'Do this task for me NOW from my boss' emails :)

  6. Blackberry has killed this habit for me. My email simply pops up as the red light flashes.

  7. Substitution. You still look down to see if the red light is flashing.

  8. First of all, I find nothing about my behaviour disturbing.
    Not sure I get the email checking comparison.(Though I too have a beeping/vibrating/flashing blackberry so I don't feel an urge to check anything until after I've been notified of something I'm not sure I'd equate email with reward.)

    But in keeping with your analogy, is the story that people will check their email more often and/or more "religiously" when experiencing intermittent reinforcement than say, if they knew that every time they checked they'd receive a new email? Is that what you mean by "most successful reinforcement schedule"? It was most successful at accomplishing what exactly? Developing habits?

    I've heard of something similar before, but it's kind of hard to understand why intermittent reinforcement would be more habit-making than constant reinforcement.
    Wouldn't you play the lotto more frequently if you knew that each time you did you'd win something?

    I can imagine that if the reward was intermittent, we could then consider the development of some sort of superstition or ritual behaviour in an attempt to explain (and control) the unexplainable intermittence. Then this superstition and ritual gets validated and strengthened with each reward, regardless of the number of non-reward instances. But do mice develop this too? Is this even a factor?

  9. Drei -

    "Reward" in the behaviour theory sense, interchangeable with reinforcement.

    A successful reinforcement schedule is one that consistently elicits the target behaviour even, and especially, in the absence of reward/reinforcement.

    Yes, people would play the lottery more if they knew they would win every time. The principle of intermittent reinforcement explains why people continue to engage in a certain behaviour (e.g., email checking, playing the lottery, playing slot machines) in the absence of reward.

    Intermittent reinforcement is more resistant to extinction (the behaviour stopping when the reward stops) than is continuous reinforcment. Animal trainers use this all the time. If you reward continuously, as soon as you stop rewarding, the animal will stop the behaviour. The animal learned that the reward always follows the behaviour, and since the reward didn't follow this time, the animal thinks that relationship is over. If you reward intermittently, extinction takes much longer because the animal doesn't immediately think it will never be rewarded again just because it wasn't this time.

    That's the underlying principle of intermittent reinforcement: you weren't rewarded this time, but you could be next time.

  10. Love this post! Nice to have an explanation as to why obsessive e-mail checking is so gratifying.