I’m not an expert at archery, but there’s a way I can guarantee I get a bull’s eye every time. It’s simple. I shoot the arrow at a large target, one I can’t possibly miss. Then, I walk up and draw a bull’s eye around the arrow.
We laugh at how ridiculous this is, but this sort of error occurs every day in peer reviewed studies published in prestigious journals. Called the Sharpshooter Fallacy, it’s all too common, so it’s important that we be able to recognize it.
Statistics have always been used to obscure the truth. There’s the famous quote by British prime minister Benjamin Disraeli, and popularized by Mark Twain: “There are three kinds of lies: lies, damned lies, and statistics.” Sadly, nothing has changed. In fact, it’s gotten worse.
The problem is that statistics isn’t a popular subject, and many people happily avoid taking any classes that cover the topic. (It was affectionately known as “Sadistics” when I was in college.) We’re easily led astray by official-sounding numbers, especially if the conclusion is one we already agree with. While we (thankfully) don’t need to worry about Chebychev’s Rules, Probability Distribution Functions, or Stem and Leaf Diagrams, we should know how statistics work, and how they can be used to fool us. There are numerous ways in which statistics can be misleading. I ran into one of them while reading the news this week.
The news is everywhere—young adults are leaving the church! Survey after survey is finding that only a small percentage of kids raised in a Christian home continue on with church attendance as they grow up. It doesn’t matter if they’re on their own, or still living with their parents. Christian leaders are doing some serious soul searching, beating their breasts over what went wrong.
- The Barna Group has found that “nearly three out of every five young Christians (59%) disconnect either permanently or for an extended period of time from church life after age 15.”
- A survey by LifeWay Research found that “seven in ten Protestants ages 18 to 30 who went to church regularly in high school quit attending by age 23. A third of those had not returned by age 30. That means about one-fourth of young Protestants have left the church.”