Correlation is Not Causation
As a Business Analyst, the mantra I try to remember every day while I try to figure out how the company is doing, why it is the way it is, and what we can and should do, is that correlation is not causation. Sometimes logical fallacies are put forth by well-intentioned people and actions are taken based on that faulty logic which could have been more effective if superseded by a nice dose of common sense. In our LDS setting, let me put forth a few examples, having to do with conversion and retention.
1. 50% of youth are inactive before they reach the age of 12, Therefore Primary Teachers are the most important calling in the Church.
Ok, so here’s the basic common sense test. While it may be true that children under the age of 12 represent 50% of the youth who go inactive, who ultimately decides if these kids go to Church? Do they, or do their parents? Parents right? So if parents are the ones in charge of minors, then that means 50% of the youth in the church are going inactive before the age of 12 because 50% of the youth’s parents in the church are either allowing their youth to go inactive, or are going inactive themselves. Thus Primary Teachers are not the most important calling in the church. Parenting is, and teachers who train and encourage those parents at church would supersede other roles in the church, or should.
2. 1 in 6 investigators taught in a member’s home gets baptized, so missionaries should teach all of their investigator’s in members homes.
Common Sense Test: Ok, So in a normative situation, what investigators are taught in member’s homes? Member’s friends who are close enough relations to the member to have been in the member’s home. Why would these people be more likely to be baptized? Because of their close relationship with the member, or in other words, because they already have a social foundation in Mormonism. So if you take a complete stranger into a member’s home, the probability of them getting baptized may not see such an exponential increase. We are better served encouraging our members to be open honest and most importantly good neighbors, rather than weaken this statistic by trying to force situations.
3. Members from a distant country who go to the temple are more likely to stay active, so we need to build more temples that are closer to members so they will stay active.
Common Sense Test: Members who are willing to sacrifice and invest of themselves to go a great distance for the church are more likely to stay active in any case that people who are unwilling to sacrifice and do not invest themselves. By making the investment required exponentially smaller, this statistic’s effectiveness will also become smaller. Thus we are better served in finding appropriate ways for each member to have an opportunity to invest themselves in the Gospel experience, whether that be temple work, the cannery, teaching primary, or any other calling we can extend.
So it seems the real answers to the three corollaries above are 1. Nourishment by the good word of God. 2. A Friend and 3. A Calling.
Go figure.
Your second correlation used to drive me crazy as a missionary. Your common sense analysis of that situation is spot on.
From the title, I thought this was going to be a post ripping the correlation committee :)
Comment by Jacob J — October 6, 2008 @ 10:20 pm
Your last two paragraphs are so perfect that I’m reluctant to comment and detract from them.
Here’s a possible alternate/additional explanation for #2: friends of members are a selected and self-selective group—people typically choose friends who are like them (i.e., Mormons hang out with people who are “practically Mormon”, and non-Mormons who hang out with Mormons have already passed the “you guys are weird” test). Missionaries, on the other hand, teach whomever will listen and some investigators are disappointed when they find out that the church isn’t filled with enthusiastic young men.
I don’t think that disagrees with your main point.
Jacob J: that’s funny, because from the title I thought this would be in defense of the correlation committee!
Comment by BrianJ — October 6, 2008 @ 11:06 pm
Amen.
Comment by Stephen M (Ethesis) — October 7, 2008 @ 5:31 am
FINALLY!!!
You are spot on. I would like to recommend you to be the next Stake President in the Kalamazoo Stake.
Comment by Eric Nielson — October 7, 2008 @ 5:49 am
Jacob and Brian: It was D. Michael Quinn who said “…correlation…is…[great]…” (I kid)
Eric Nielson: Everytime I see the word Kalamazoo, I think “Skittamarink a dinky dink, skittamarink a doo”
Comment by Matt W. — October 7, 2008 @ 6:33 am
Does anyone actually teach that primary teachers are the most important calling? I mean, other than just saying it. Are there wards out there that actually make their best people primary teachers? I’ve never heard of such a thing.
Good post, though. One that I was big on during my mission involved the fact that the white bible says that member referrals were the most effective way of finding people to teach. What it doesn’t say is that most of those referrals are probably voluntarily given. As it turns out, hounding members into giving up the name of someone who has zero interest in the church isn’t very effective after all. Who would of thought?
Comment by Eric Russell — October 7, 2008 @ 8:04 am
Eric R.: The Primary statement was made by a counselor in a different Stake. He was responding to someone else saying that seminary teachers were the most important calling because 99% of Institute graduates stay active. (The problems with that statement abound, in and of itself, becuase seminary teachers don’t teach institute) It was the initial idea that got me thinking about this post.
When I was on my mission, something I fought to change was the idea of buttering members up for refferals. Even Members in the philippines have been told the Fruit in Lehi’s dream was sweet about a thousand times. They know you are a missionary, and they know it’s your job to ask for referrals. Just ask directly once, and make yourself available to receive. If you are a good missionary and left in an area long enough so people get comfortable with you, referrals will come. I think the most effective thing a missionary can do to get referrals is to bear his testimony on fast sundays and get into the houses and lives of every member in the ward. Sorry for the Soapbox, but let’s just say it’s my way of agreeing with you.
Matt
Comment by Matt W. — October 7, 2008 @ 8:32 am
The Church collects a wide variety of statistical information … which is apparently then used to make managerial decisions about programs and policies. Except that the whole process is hidden behind a managerial veil. We don’t know who is doing the analysis, how such data is being analyzed, and how decisions are formulated.
My suspicion is that most findings and recommendations based on LDS statistical data get filtered through strong existing biases. In other words, data that lead to unwanted conclusions don’t get passed along very often. I’d agree that even common sense observations sometimes don’t penetrate that existing bias filter.
Comment by Dave — October 7, 2008 @ 8:54 am
Good points Dave.
Comment by Geoff J — October 7, 2008 @ 9:12 am
Dave: trust me, the same things happen in business. No one wants to tell the CEO that the new Product X we were all so self-assured about is driving people away from the company. One of the biggest issues I run into is fear of bad news and spinning that data to be something other than it is. Another issue that happens often is business analysis is people get so focused on one possibility that they jump the gun before testing the hypothesis because their experimental analysis was so exciting. It’s not how the item analyzed is doing that analysts mostly get wrong, it’s why it is doing whatever it is that it is doing, and what that therefore should mean to us. It comes from not being able to control for or understand all factors that are at play in the marketplace. The same thing is a major issue in the government as well.
Comment by Matt W. — October 7, 2008 @ 9:17 am
We recently had missionaries over for dinner and they started their, “Your friends really need the Gospel” lead-up. I was a bit tired and frustrated by other things and so I cut them off and said, “Elders, I realize that you are concerned about my friends, but let’s agree that they are my friends. I love them. If there were any way I could help them move closer to Christ, I would have already done it.” It’s not entirely true—I could do more—but the sentiment was right.
Comment by BrianJ — October 7, 2008 @ 10:01 am
I’ve usually heard it as Home Teacher is the most important calling.
Comment by Clark — October 7, 2008 @ 10:17 am
I think some of this analysis is striking out against the problem of superlatives in the church, and how they are used improperly regarding any topic that comes up.
If a person is giving a talk or a lesson on any given principle or topic, there is a temptation to take emphasis too far and turn that principle or topic into “the most important.”
It’s an inclination we have to be careful to avoid.
Truth is, whatever is “most important” at any given time for any given problem is probably unknown. We can look at statistics and percentages about inactivity – but it is unlikely there is a single explanation for all of those cases.
Comment by danithew — October 7, 2008 @ 10:31 am
danithew,
That may be the most important comment ever made at this site.
Comment by Jacob J — October 7, 2008 @ 10:50 am
danithew, In business, companies don’t want every customer, just as many customers they can get for a reasonable cost. So in Business, generalities are fine because insofar as statistics and percentages get to the majority, which is good enough. It could be argued that in religion, the majority is not good enough, and that we need to reach out for each and every one.
But still, if one thing can and does make a significant difference for a large subset, I am all for it.
Also, a point of clarification for all. I see that BCC linked to me in regards to church wide policy making. I want to say that In the above statements, I am grateful fot the data provided. Where we get in trouble is what comes after the “therefore”. The facts above are all true, it’s the connection of these facts as reasons that is problematic. It’s kind of like saying the only purpose of the law of chastity is to prevent STDs.
Comment by Matt W. — October 7, 2008 @ 11:05 am
Some years ago I sat in a meeting listening to a leader talk about the inactivity of our young adults. He was basically saying that those who work the the young adults must be doing something wrong. Of course, all of us in the room worked with young adults. I sat there feeling a bit irritated, but not guilty for any bad efforts on my part, when a great moment happened. A man in the congregation stood up and said “the question isn’t ‘what are we doing wrong with our 18-30 yrs olds,’ the question should be ‘what went wrong from ages 0-18 that causes them to go inactive when they turn 18.'” The speaker shook his head slowly in agreement and just sat down.
Comment by Hal — October 8, 2008 @ 9:26 am
Here’s another correlation/causation issue that bothered me back when I was in YM: “99% of eagle scouts go on missions, so get those merit badges if you want to be a good missionary” (BSA is great and all, but c’mon)
Comment by rd — October 8, 2008 @ 10:22 am
Well now my comment just looks silly.
Comment by BHodges — October 8, 2008 @ 5:34 pm
Sorry about that BHodges. I deleted the spam and left you hanging.
Comment by Geoff J — October 8, 2008 @ 5:41 pm
My wife and I discussed this a bit and I wanted to state that if controls were put in place, it may very well be that temple worship and members having investigators in the home do produce these exact activities, but my assumption was these controls were not in place. If anyone with access to church data wants to correct me, I am all for it. AND I would take half pay to work on church data, For the record.
Comment by Matt W. — October 8, 2008 @ 5:47 pm
AND I would take half pay to work on church data, For the record.
If only the church weren’t so used to offering NO pay, that might seem like a better offer. {g}
Comment by Jacob J — October 9, 2008 @ 10:49 am
If I could do it from texas without having to quit my current job, I’d do it for free, Just to know.
Comment by Matt W. — October 9, 2008 @ 2:52 pm
Matt, your conclusion is striking, as I’m sure you intended it to be. Sometimes, I wish we just accepted what the Prophet and apostles lay out for us and spend our efforts doing those things. I don’t mean to suggest blind obedience; I’m talking more directly about not trying to create our own programs and extrapolations – since those generally result in twists like those you highlight here.
Thanks for this post. I am trying to figure out how to work it into the next training I do in my calling.
Comment by Ray — October 10, 2008 @ 11:17 pm
Dave
You would be very interested to know how, when, and by whom many of those “church statistics” are compiled. I’ve been fortunate enough to assist in one project analyzing “Preach My Gospel”‘s effectiveness. Trust me–everything is there, none of the bad things are left out.
Comment by Tom Rod — October 11, 2008 @ 9:59 pm
Tom,
Yes we would definitely be interested to know how, when, and by whom these statistics are compiled. Can you fill us in on some details?
Comment by Jacob J — October 11, 2008 @ 10:10 pm
Tom, if you are interested, send me a write up and I’ll put it up as a guest post. Everyone here would be very interested.
l m w i t t e n @ p r o d i g y . n e t
Comment by Matt W. — October 12, 2008 @ 10:59 am