Survey Results- Part 3 – Engagement and Outcomes
Continued from here.
Well, we’ve talked about engagement before, so I don’t want to spend too much time on it, especially as there have not been that many dramatic changes in the results, even though the questions I used this time are a little different.
To begin, here is the basic rundown of results:
As you can see, we’re still pretty in sync with our last survey. We still need to work on all the same things and are still positive in all the same areas. This time around, I added an Average Score of Total Engagement which works just like the Average Discipleship Score from the previous post.
The breakout across our 1322 respondents is as follows:
As we’ve already discussed, there is a pretty strong correlation between Discipleship and Engagement.
In other areas though, there seem to be more discrepancies.
While women clearly dominated on the strong side of the engagement buckets on the discipleship section, the results are not as clear in a similar chart here, however, when we break them out based on average scores, men clearly lose out.
So women are more engaged and better disciples than men. More on that in a minute, but first let me tell you about desired outcomes.
While Gallup suggests outcomes like donation amount as share of wallet and hours of church service given, I did not really think those were appropriate for the LDS audience and narrowed the desired outcomes to 3.
As can be seen, the most limiting factor here is the rather specific question on missionary activity. I almost excluded it from the outcomes, but decided not to, as I do think sharing our faith is a worthwhile pursuit.
Anyway, so let’s take a look at the correlation between discipleship, engagement, and outcomes from the perspective of outcomes.
As expected, higher engagement and higher discipleship both lead to better life outcomes within our sample set. With that in mind, and with the gender correlations we’ve seen up to this point, the following should come as no surprise.
While this chart is pretty explicitly demonstrative in favor of women’s superiority, there is one nuance we miss if we don’t go to the chart.
More life satisfaction, more inner peace, but slightly less likely to have invited someone to church. To come up with a hypothesis on that, and to throw out a basic stereotype, this may just be more of a function of occupation than anything else, with women more likely to be a stay at home parent than men, and thus have less opportunities to invite others to come to church.
Implications up to now:
1. Women are more awesome than men. So the next time you hear someone say men have the priesthood as a crutch, or some other thing which sounds silly or lacks gender equality, just keep in mind that generally speaking they are right.
2. You will be more satisfied with your life and have more inner peace the more engaged you are at church. The “Sunday School” answers (Read Your Scriptures, Say your prayers, Go to Church) actually do improve your life.
Next- Why we argue about Darwinism\Creationism so much.
Very nice, but I must point out your #2 is a correlation/causation problem. You’re probably right, but you can’t tell which causes which from this data set.
I hate to pop out of nowhere and be critical, so please keep in mind that I really like your site and I’ve followed this closely. Thanks for doing it; it’s really interesting. :)
Comment by Melynna — May 14, 2010 @ 10:13 pm
[editor’s note: typo corrected, thanks!]
hard to really credit internet surveys enough to say that women are “more awesome” than men. I hope that’s just your light-hearted tone instead of a thoughtless phrase. But let’s be honest (Mormons should be): women are pretty awesome.
Interesting results. I like all the colored charts, they’re pretty…
Comment by emilyf — May 15, 2010 @ 3:13 am
Melynna- While you are correct that there could be a chicken and the egg phenomena here, the correlation is consistently tight enough that for this sample set (active LDS people) that it could be said if you improve any one of the three points (engagement, discipleship, outcomes) you will probably be improving the other two. Inversely, if you attempt to improve one without improving the other two, you probably won’t succeed.
emilyf- With a sample set of 1322 respondents, standard practice says I have a 3% margin of error, but it still leaves me with women being “more awesome” (which was being light hearted, yes). In any case, women outscored men in terms of engagement, discipleship, and desired outcomes.
Comment by Matt W. — May 15, 2010 @ 8:32 am
First, I’m not getting the complaint by emilyf. Is she complaining that you said women are more awesome than men or not?
Second, I agree with Melynna about your conclusion #2.
But nice work on this again. The number crunching is interesting to be sure.
Comment by Geoff J — May 15, 2010 @ 8:48 am
“2. You will be more satisfied with your life and have more inner peace the more engaged you are at church. The “Sunday School” answers (Read Your Scriptures, Say your prayers, Go to Church) actually do improve your life”
I strongly disagree with this as an “implication”. Perhaps the opposite is true. Perhaps there are some people for whom the Church is satisfying, so they stay engaged. For others, it doesn’t do much for them, so over time they become more disengaged.
I would be considered “engaged” as far as my activity level, etc. But Church has done less and less for me as years have gone by. I could see myself drifting into less engagement. So to say I will have more inner peace by being more engaged is wrong.
Comment by Mike S — May 15, 2010 @ 8:56 am
Mike S.- Um, all the data shows a strong correlation between engagement in church, discipleship, and outcomes. You may think you are engaged at church, but not understand the actually definition from the survey. Look at chart 1 above. Those are the questions which measure engagement.
Comment by Matt W. — May 15, 2010 @ 9:37 am
I agree that there is a correlation. That doesn’t necessarily imply causation, however. I don’t know that you can make the If … Then … statement you made: IF you are more engaged at Church THEN you will be more satisfied with your life and have more inner peace.
If generalized, to me this would mean that having peace and being satisfied with like directly follows from being engaged in the LDS Church, and should be true for all people.
I do think there is a certain subset of people who happen to find that inner peace and satisfaction in the Church, so they therefore stay engaged. But I think the majority of people baptized do NOT find that inner peace and satisfaction, and therefore become increasingly LESS engaged.
It is great data, by the way, and I appreciate the work you did gathering it, analyzing it, etc. I just think it helps confirm that people find a path through life that does bring them inner peace. For those highly engaged in the LDS Church, it is this path that brings them peace. For others, it’s likely a different path. Perhaps it’s splitting hairs, but your conclusion #2 implied that the ultimate path to peace for EVERYONE is following the tenets of the LDS Church, which I don’t think is correct.
Comment by Mike S — May 15, 2010 @ 9:49 am
Mike S.- It is not the church, per se, which brings the inner peace/life satisfaction. It is the engagement. I’d refer you to Gallup press’s general studies on engagement. Which for churches they do via the Me25 study and for businesses they do via the Q12 study.
So for a study which had over 90% of it’s respondents as people who go to church regularly (more than once a month), this strongly infers that being engaged in this thing which they regularly do definitely does mean they are more happy. The Outcomes piece though, isn’t really as interesting as the discipleship pieces, which Gallup calls “spiritual committment”. The question of directional causation is a good one, but since the correlation is as strong as it is, I’d still say that improving either way should improve both ways. Since it is frankly easier to increase engagement, I’d say this is still a very good talking point.
Finally, the Gallup press has had quite a bit of success with this and has many case studies prooving their point. Check out the book “growing an engaged church” for their extensive work on this.
Comment by Matt W. — May 15, 2010 @ 10:35 am
No, Matt, you cannot conclude from this anything about what will happen if you “change” either measure for an individual. Your data set is a cross-section in time that shows how the variation in the two measures from person to person correlates, but not for the same person over time. The conclusion that if a person who is low on engagement increases his/her engagement then his/her satisfaction will go up based on cross-sectional correlation is possibly the most common fallacy in econometrics, which many, many studies fall prey to, but it doesn’t change the fact that you have not showed anything about change, only difference. The strength of the correlation here tells you nothing about the direction or about what effect changes in individuals have. All you can say is that in your data set people with high engagement have higher satisfaction. I was going to also write that you can say this relationship is statistically significant, but you haven’t shown this either. If you want, the Excel Data Analysis Toolpak will let you do this. The easiest thing to do given how you have your data structured is to use the Regression function to do bivariate OLS.
I find it a very plausible idea that people who are satisfied with church are more likely to engage, but it is certainly a case of simultaneous causation. It is interesting to note the differences in practice that each conceptualization of the causal direction would lead to: one implies that leaders should try to encourage people to engage in the prescribed behaviors and the other implies that leaders should try to make church more enjoyable for everyone regardless of engagement, which might lead to the prescribed behaviors.
Comment by Owen — May 16, 2010 @ 10:28 am
Owen, I think you are confusing engagement for discipleship and outcomes for outcomes for engagement.
Engagement as defined by the 12 questions above is more a question of connectivity, who emotionally, spiritually, and socially valuable does an individual find their relationship with the church.
More here. regarding correlation. And more extensive analyses.
When all this is said and done, I will post the data set and invite any takes to guest post any conclusions they’d like to share. Stick around.
Comment by Matt W. — May 16, 2010 @ 11:29 am
For those who don’t want to buy the book. This Parish did a great outline of it here.
Comment by Matt W. — May 16, 2010 @ 11:41 am
Matt,
From their comments, there is no reason to suggest that Mike S & Owen are confusing outcomes for discipleship and engagement. The point is, you are very likely measuring a self-selected group which isn’t being taken into account in your assertion that increasing engagement will increase satisfaction outcome.
For example, try this thought experiment: start with a group people of varying engagement and satisfaction. Those with high satisfaction are likely to increase their engagement and those less satisfied will decrease their engagement. Taking a snapshot of this group at any time after time zero will give the same correlation you are seeing, but with satisfaction driving engagement rather than engagement driving satisfaction. This doesn’t mean that engagement can’t drive satisfaction, but such a ‘Sunday School Answer’ effect is not demonstrable in these data.
I admit that I haven’t read the Winesman book, but there really isn’t any indication from the posted outline that he can reliably distinguish between them either – even if it is a fresher way of looking at things.
Great graphs, BTW.
Comment by Northboundzax — May 16, 2010 @ 2:30 pm
I’m interested in the “Women Win” on so many of these. I wonder if women (especially in this Church) are just programmed to Everything’s Great responses. Since the results are _so_ skewed one direction, it almost makes me think that females are just more inclined to put an enthusiastic gloss on things.
I’m not great at math, but it looks like the responses are always about .25 different. That again makes me think maybe it’s just a male/female response difference, rather than a Women Win at Discipleship thing — not to disparage my sex, of course.
When they make me General RS President, my first talk will be on how women and men should speak of each other. It weakens us women to vaunt ourselves by talking down someone else. It does not make men more humble to rip on themselves or over-glamorize womanhood, either. Neither sex is better than the other. None was the “crowning” creation. We are equals in our heavenly awesomeness — different — but equal.
It’s time for men and women to stop wasting our time on who is “more righteous” or “in charge” and focus on the ultimate relationship/leadership goal: oneness. When we’re truly one, those labels have no meaning.
Whoa! That was a rant! And it wasn’t even we-have-too-many-meetings related: ha!
Comment by Jenny — May 16, 2010 @ 6:22 pm
Zax- You seem to be suffering the same problem.
Jenny- I’d rather go with women are more awesome than women are “programmed”. Sorry. Mormon women I’ve met are not “programmed”. I think that’s insulting.
Comment by Matt W. — May 16, 2010 @ 7:27 pm
If by ‘same problem’ you mean your conclusion overstepping what the data can tell you, then yes.
Comment by Northboundzax — May 16, 2010 @ 7:51 pm
Matt: It is not the church, per se, which brings the inner peace/life satisfaction. It is the engagement.
It seems to me that the objection is that you keep implying causation with comments like this when all your data shows is correlation.
Comment by Geoff J — May 16, 2010 @ 9:43 pm
Jenny: I wonder if women (especially in this Church) are just programmed to Everything’s Great responses.
And I now wonder if you are a misogynist. (But of course I wonder that in the nicest possible way…)
Comment by Geoff J — May 16, 2010 @ 9:46 pm
I wish you’d give Jenny (#13) a break. Even if your data supports that “women are awesome”, what Jenny is saying is that repeating it again and again as leaders of the Church so often do, doesn’t help men change and the current limiting of women in the real leadership positions doesn’t make a lot of sense if “women are awesome”. If women are awesome, then they should be leading us. And if they are really leading us now anyway in a default way, which a couple of church leaders implicated in General Conference in April, then us men are really fooling ourselves and is this deception making us all confused? Instead of “women nurture, men provide”, from your implications that “women are awesome” we should be preaching “everyone needs to learn how to nurture.”
Comment by kevinr — May 17, 2010 @ 11:27 am
Geoff- That’s because proving causation requires multiple experiments rather than a singular survey. I am really relying on Gallup’s research for the causative nature of this, which they did over a 5+ year period and through many test cases. But it is fair to say that if you don’t buy into their research, mine is even more invalid.
KevinR- I did give Jenny a break. I didn’t argue any of those points, I just said calling women programed is a untestable assertion with no factual backing, whereas my statement of “women are awesome” is in this instance totally backed by data. If that pisses you off, wait till you see how women responded to questions about gender equality, heavenly mother, polygamy, etc.
Comment by Matt W. — May 18, 2010 @ 3:40 pm
Wow. You boys are kind of mean.
I do agree programmed was a bad word choice on my part tho; perhaps expected would have been better. So was “more awesome” on yours.
Let me clarify: all this data does is show that women *report* thier engagement to be higher. I think women in our church do report higher than men. It’s a gender-church-cultural thing IMO.
Not only that, but female engagement in our church is sometimes easier to achieve than male engagement. When I visit teach, if things are busy all I have to do is drop a card in the mail or make a call. DH doesn’t have that luxury and obviously can’t report his engagement factor as high as mine, though his intentions are just as good. I’m not whining about the status quo. I’m just saying the requirements for engagement aren’t the same–how does one take that into account? I don’t know, myself.
It seems odd that the data seems to be skewed the same direction to almost the same degree in each response.
Why must we split these types of studies down by gender? Is it a competition? Even if you don’t feel that women are oppressed somehow, this type of split implies that we do: “Women may not be in charge, but they make up for it by being ‘more awesome’.” I find this insulting not only as a woman, but dangerous to my son who might infer from this type of talk that being male is ‘less awesome’ and therefore bad. Both genders are equally awesome. It weakens us to build up one by degrading–even if only by implication–the other.
Maybe there are more useful ways to break down this data. Like how engaged do singles or marrieds or widowers or minorities or new converts feel?
I’m sorry I’m not smart enough to know what a misogynist is. If it means I’ll say something when you bash men and say something when you bash women, then yes, you called it. If it means woman-hater, maybe you should sit with me next Sunday in ward council. ;) it’s lame to name call after reading only 10-12 lines on a blog.
Comment by Jenny — May 19, 2010 @ 11:18 am
Jenny-
” I’m just saying the requirements for engagement aren’t the same–how does one take that into account? I don’t know, myself.”
This is not correct. In this study, the requirements for engagement are EXACTLY the same. These are the only requirements for engagement taken into account. Here was the break out by gender.
Anyway, this set of engagement questions seems to always skew towards a better response for women than men, in religion or business
Maybe I shouldn’t have said “More awesome”. The Question is really, why are women more engaged, or why are women more religious.
To quote the linked article, the possible reasons are
“Maybe there are more useful ways to break down this data. Like how engaged do singles or marrieds or widowers or minorities or new converts feel?”
I did this, there’s basically parity between the segments as far as married /single. For converts/new members, see here. Results did not change much from there. So I din’t post results where there was really nothing new to report.
In terms of engagement, women, as a unit, are more awesome. Men may be more awesome in other areas. I am not degrading men. If we can’t acknowledge there are differences, we can not work towards realistic solutions.
Comment by Matt W. — May 20, 2010 @ 7:07 am
Jenny: I just thought I’d throw out that I 100% agree with you. I think you are a very honest.
My personal feeling is that sometimes ego taints our views. Growing up I was one of those people (like you worry about your son) that actually felt men were inferior to women in the church. I always felt women were viewed as more righteous, noble, holy and less egotistical or sexual?
So I totally understand that. Further, I think there is a tendency for some (of course I don’t pretend to KNOW everyone’s intentions) to throw out comments about great women are to protect their ego and rationalize away things that might come across as misogynist, or not fair to women in their mind. Kind of like over compensating? I think to throw out how great women are can come across trivial.
Again, just wanted to say (if I can with the 10-12 lines you wrote initially) that you seem very nice and honest and appreciate your insights. :)
Comment by Kevin — May 20, 2010 @ 10:48 am
Matt (or other blog writers): I actually come across this blog from time to time when stressing about doctrinal issues I suppose?
I was goggling Psalm 137 (the one that ends with “dashing infants upon the rocks”).
I heard a talk from an Islamic Council (some group in So. Cal) talking about how Islam is not misogynist or violent and I took issue with that reading some of the verses myself. I have not read the whole Koran but have one and have read segments. Seems far from violent overall, but definitely has its violent overtones toward the second half especially.
I didn’t have the courage to mention this to the panel, but luckily someone else did, to wish the Islamic Council replied about the violent passages of Jesus (sell your clock and buy a sword) and came not to bring peace but the sword. (The latter I think is metaphorical and I have little problem, but the former seems a little like its at least advocating self defense to me even though Jesus rebukes in the other gospels?)
He also mentioned Psalm 137 which I think is horrific at the end.
Someone on Times and Seasons wrote a piece about how it was disturbing and said some nice words but it wasn’t really satisfying.
I guess I would be part of the group in the survey that is not active and not very involved in discipleship, but is very conflicted! so I got the worst of both worlds maybe :P
Anyway, I was interested to know if you have done a thing about violence in the Old Testament (maybe New) or Islam’s violence (related to your post a week ago?)
Or maybe you could at least address Psalm 137 and the calls to death in Leviticus?
I miss the days where certain belief was not something that I Felt was unattainable. Kind of sad to me in some ways that the highest discipleship correlated with uncertainty. But maybe that’s just online as others said.
Nevertheless, good work with all the surveys. Regardless of interpretation, thanks for contributing to the knowledge base we have on here!
Comment by Kevin — May 20, 2010 @ 10:57 am
The data you’ve posted is very interesting. However, I tend to agree with Owen – your conclusions are not fully supported by the data. Perhaps a 2sls regression – to prevent simultaneity bias – would be helpful.
Comment by CJ — May 20, 2010 @ 3:54 pm
Jenny, sorry we are so mean, you make some important points.
Matt, On the engagement questions, it seems like engagement is not really a strong function of the person in question but rather their ward or their perception of how they are treated. Whether you feel appreciated, have a friend, have your spiritual needs met, have opportunities to learn/grow, etc. are not necessarily up to you. It seems like this data may support the idea that the church does a better job catering to the needs of the women in the church. What are your thoughts on that front?
Comment by Jacob J — May 22, 2010 @ 11:30 am
Jacob: the only issue with that hypothesis is again that these engagement questions , or at least a very similar set of them, are true across the Catholic and Protestant churches as well. Also, women typically score higher on these questions in relation to workplace engagement as well, generally speaking. So all churches (generally speaking)do a better job of catering to these needs of women, and the same is true of workplaces as well.
Maybe women (generally speaking) are more predisposed, as Jenny suggested (and I shot down), to have more positive feelings on these topics. Guess I should apologize to her.
I am tired of writing “generally speaking”
Comment by Matt W. — May 22, 2010 @ 12:48 pm
Hey Matt,
Any thoughts on Psalm 137? You can email me if you’d like rather than do a whole post just dedicated to my request. :P
Comment by Kevin — May 22, 2010 @ 1:30 pm
Kevin:
Honestly, I am no old testament expert. I’d suggest you go to either the blog feast upon the word or ask Nitsav at Faith Promoting Rumor. Both are awesome.
Comment by Matt W. — May 22, 2010 @ 2:06 pm
Matt (#19): I am really relying on Gallup’s research for the causative nature of this
My memory must be failing me. Could you remind me of the specifics of this Gallup research? Also it is not clear to me how that research is helping your feel confident about making causal claims regarding this survey.
Comment by Geoff J — May 22, 2010 @ 10:24 pm
Jenny (#20): Wow. You boys are kind of mean.
Word. You know what they say about boys: “Snakes and snails and puppy dog tails…”
I do agree with your sentiments about the “more awesome” genre of comments that are prevalent in our culture though (see Mother’s Day talks for another fine example). The practice seems like a bit of a consolation prize to the brilliant women who are baffled by the fact that they aren’t allowed to be really in charge in the current church. But of course there are multiple feminist Mormon blogs who talk about that subject ad nauseum so I won’t belabor it here.
Comment by Geoff J — May 22, 2010 @ 10:26 pm
Geoff- see the links in comment 21.
Comment by Matt W. — May 23, 2010 @ 6:45 am
Matt,
Thanks. I’ll check those blogs out.
Comment by Kevin — May 23, 2010 @ 8:16 pm
I agree with Jenny’s sentiments. Women are taught to be more agreeable and happy.
However, I wonder if some of the “women win” may also come from the structure found in Relief Society. I wouldn’t say that women in the church are more spiritual than the men – perhaps they just have a better organization within the church? I have been on and off active for 8 years, but have always had access to church groups through relief society, whether it be a book club or lunch. I am engaged with other women and can easily form a bond/friendship – Questions asking about if people care about me and if I feel important – would be directly addressed by these types of gatherings. To my knowledge men in the church do not this same type of opportunity – they engage with other members either at church or a ward activity – not very many activities that are just male attendees, therefore the bond/friendship is less with men??
Comment by Marie — May 26, 2010 @ 12:16 pm
Women are taught to be more agreeable and happy.
Seems to me that a lot more effort is put into teaching/exhorting men to be agreeable and happy at church than women. I suspect that is because women are evolutionarily designed to be more socially savvy and connected than men are.
Comment by Geoff J — May 26, 2010 @ 12:38 pm
Geoff: I think that is closest to the truth, since women win at work and in other churches as well. Marie, I don’t think men, in general, want those types of activities/obligations.
Comment by Matt W. — May 26, 2010 @ 12:54 pm