When ‘common sense’ programs backfire
What do these two teenagers have in common? Kayla is a junior in high school and is spending several weeks feeding, caring, changing diapers, and acting as a mother to Jayden. Kayla’s schedule is upended, and she feels the fatigue and frustration that caring for an inconsolable infant can bring.
But Jayden is abruptly taken away from Kayla after just a few weeks.
Why? Because Jayden is a robot baby, and Kayla participates in a school program that shows teenagers firsthand what it’s like to have a baby. This program is so popular that 89 countries around the world use it.
Now let’s look at Colton. He is also a junior in high school, but he’s considered “at-risk” due to his truancy and shoplifting arrest last year.
Colton is spending the day at a county prison, not for a new crime but as part of a program at his school. He hears about the difficulty and horror of prison life from real convicts. They don’t hold back or sugarcoat what would happen if imprisoned. The prisoner's goal is to make prison seem so bad that he is “Scared Straight.”
These programs make intuitive sense. If you realized how difficult life is as a single mom, you'd take steps to avoid teen pregnancy. If you understand prison is awful, you’ll avoid situations, friends, and activities that get you arrested.
But there’s one problem—both programs don’t work.
They lead to increased rates of teen pregnancy 1 and incarceration rates.2
Knowing these programs fail, it’s easy to think of why.
Taking care of a robot baby leads Kayla to enjoy being a mom. She realizes she can handle it. Unconsciously or consciously, Kayla may stop taking steps to avoid pregnancy.
Colton is scared by some prisoners but, at the same time, respects some of the inmates. He may even view some prisoners as stupid or dumb…no wonder they got caught. Colton believes he’s smarter than the prisoners and won’t make those dumb mistakes.3
What’s at play here?
As humans, we are great storytellers. Telling stories is fundamental to how we understand the world and how we explain it to each other. We can look at a series of facts and string together logical, intuitive reasons to justify how it is related.
The stronger the narrative, the more we believe it is true and do not question its conclusion.
This is called the narrative fallacy. It’s especially seductive when we create stories on seemingly obvious and intuitive explanations.
The stronger the story, the stronger the belief.
But how can you avoid doing the same thing?
To discover firsthand how this works, you must do a little homework and run an experiment.4 Your experiment can be as simple as an A/B test with only two groups. Ideally, done by someone with no vested interest in the outcome.
Randomly assign half to the test, and leave the other half untouched as a control group. The key here is randomization. Note that you must have the same types of people who make up each group.
Then measure the difference between your test group and the control.
Collecting the data is easy, but it takes courage to accept negative results.
Perhaps your experiment may prove the program or “story” has no effect, and there is no difference with the control.
Or you may get a negative effect, as the robot baby study showed.
A teenager like Kayla who participates in the program was twice as likely to have a child by 20 as someone who never participated.5
Just because something sounds like a good idea, makes obvious common sense, and has a strong narrative doesn’t make it work.
The takeaway:
What are you or your organization doing that is based on a strong, compelling narrative but has never been questioned for effectiveness?
The narrative may be wrong. Test it.
Notes:
See this Lancet study for more details.
The scared straight program touted an 80+% success rate for years. But the “success” was based on a survey. A voluntary four-question survey parents completed shortly after the prison visit. The parents who didn’t respond were missing in the data and not included in the analysis.
A teenager like Colton could be 28% more likely to commit a crime after participating in Scared Straight. For more info on the failure of Scared Straight read Matthew Syed's, Black Box Thinking, pg 159-165.
If you can easily think of obvious reasons and a narrative why it doesn’t work, you absolutely need to test. You may have just used the wrong obvious narrative at the start.
Participants were also 50% more likely than the control to have an abortion by age 20. See this Lancet study for more details. The study looked at an Australian version of the program called Baby Think It Over. In the US it is called RealityWorks.