orgtheory.net

Facebook field experiment shows strong ties affect voter turnout

The most recent Nature features an article by a team of political scientists and network scholars who did an experiment using Facebook to show that strong ties influenced voting behavior in the last election. You may say, so what? We’ve known for a long time that social influence operates through strong ties in interpersonal networks. That’s not a new insight.  But I think the study is innovative for a couple of reasons. The first is that the impact of of using direct messaging through Facebook was substantively significant  – that is, just messaging people reminders to go out and vote increased the likelihood that the person would vote – but that the larger effect was transmitted indirectly via social contagion. Consider the setup of the experiment.

To test the hypothesis that political behaviour can spread through an online social network, we conducted a randomized controlled trial with all users of at least 18 years of age in the United States who accessed the Facebook website on 2 November 2010, the day of the US congressional elections. Users were randomly assigned to a ‘social message’ group, an ‘informational message’ group or a control group. The social message group (n = 60,055,176) was shown a statement at the top of their ‘News Feed’. This message encouraged the user to vote, provided a link to find local polling places, showed a clickable button reading ‘I Voted’, showed a counter indicating how many other Facebook users had previously reported voting, and displayed up to six small randomly selected ‘profile pictures’ of the user’s Facebook friends who had already clicked the I Voted button (Fig. 1). The informational message group (n = 611,044) was shown the message, poll information, counter and button, but they were not shown any faces of friends. The control group (n = 613,096) did not receive any message at the top of their News Feed.

People in the social message group were 2% more likely to click on the I Voted button than people who merely received the informational message. Both groups, of course, were more likely to vote than the control group.  Moreover, the effect was accentuated by your close friends receiving the social message. For each close friend who received the social message, a person was .22% more likely to vote.  In short, the study shows that a simple message that conveys friends’ voting behavior significantly increases your own likelihood of voting.  The increase in probability may seem small, but when the effect is accentuated by the number of friends who report voting, it can lead to noticeable differences in aggregate voter outcomes.  The results “suggest that the Facebook social message increased turnout directly by about 60,000 voters and indirectly through social contagion by another 280,000 voters, for a total of 340,000 additional votes. That represents about 0.14% of the voting age population of about 236 million in 2010.”

And this brings up the second reason this study is innovative. The study conducted this experiment using 61 million Facebook users as test subjects! That’s an enormous project. I’ve never heard of a field experiment carried on that scale. The analysts can precisely identify what impact their experiment had on the election voter turnout because they had such a large pool of test subjects. It’s really amazing that social science can engage in an experiment of this size.

Here’s the abstract:

Human behaviour is thought to spread through face-to-face social networks, but it is difficult to identify social influence effects in  observational studies, and it is unknown whether online social networks operate in the same way. Here we report results from a randomized controlled trial of political mobilization messages delivered to 61 million Facebook users during the 2010 US congressional elections. The results show that the messages directly influenced political self-expression, information seeking and real-world voting  behaviour of millions of people. Furthermore, the messages not only influenced the users who received them but also the users’ friends, and friends of friends. The effect of social transmission on real-world voting was greater than the direct effect of the messages  themselves, and nearly all the transmission occurred between ‘close friends’ who were more likely to have a face-to-face relationship.  These results suggest that strong ties are instrumental for spreading both online and realworld behaviour in human social networks.

Written by brayden king

September 13, 2012 at 3:03 pm

19 Responses

Subscribe to comments with RSS.

  1. One thing I don’t get: the timing. So lets say I signed into facebook at 7AM, and I got the social message. How would any of my friends profile pictures show up as already having clicked the “I voted” button? They couldn’t. So the sequence of events matters here (timing of signing in, and doing so repeatedly, and seeing the emergent quality of your close friends having voted. Another way of putting this is that the effect of your friends voting upon you is conditional on the fact that you’re embedded within a group of people who vote. So if my close friends are unlikely to vote I’m less likely to see them show up on my social message as voting. They can’t then influence me. But it could be that they’re just less likely to vote, no? I guess I should read the paper more closely (I skimmed last night).

    The effect sizes are tiny, and the variance explained is really low. So… they had a population of about 62,000,000, of which 60 million are in the social message group. If they’d had a sample of 6,000,000 they probably wouldn’t have found anything. That doesn’t mean their observations aren’t real — it means that the potential influence upon voting is really small and their sample is really big. But it’s not inconsequential. As they note, increasing the likelihood of voting by tiny amounts can mean huge differences in turnouts. But, we don’t know if it matters for outcomes. So we don’t know if people who aren’t voting but who are on social media sites are just random (in which case there would be no real influence upon an election), or, patterned (say, more likely to belong to a party, in which case it really could influence an election). In other words, there is no indication at all that this “improves” our democracy, because only a few more people are voting and we have no idea if their voting gets us any closer to the “real” outcome preferences of Americans.

    That said, I still agree, it’s an interesting idea. We learned that we’re influenced by our friends, which doesn’t mean much to me. But still… I’m with you. It’s cool. Probably should have read more carefully before commenting. Will do later in the week…

    Like

    Shamus Khan

    September 13, 2012 at 3:40 pm

  2. I think the real strength of the study is the potential it offers for doing massive online field experiments. As more and more people engage in social interaction online, we’ll be able to do more realist field experiments of this type, and I imagine that over time the precision with which they’re carried out will only improve.

    That said, I agree that there are some problems in identifying causality in this study. I had some similar concerns to you. Another concern I had was about the social desirability of saying “I Voted.” Could it just be that people whose friends say they voted feel that they should say they voted too? If there is a strong, emerging norm in your personal network that voting is a good thing, you may simply say you voted just to fit in. I’m not sure how much of the variance can be explained by this sort of thing. These problems plague most network methods however, and so I’m not sure if this study is any worse than studies based on survey results. It’s just that it’s easy for us to pick on because we’re overly-familiar with Facebook and how social dynamics on Facebook work.

    Like

    brayden king

    September 13, 2012 at 4:32 pm

  3. I don’t believe the result at all. A .22% increasing in voting probability is a ton – campaigns spend enormous money to get out the vote in far more sophisticated ways, and can’t change voting probabilities that much. All we know is that the probability of clicking I voted, in a way shown to your friends, increases .22%. We know nothing about actual voting behavior. But this is doable! They have location data on facebook. Do the same experiment, but randomize across counties with and without treatment, then look at actual vote outcomes. The size of the supposed treatment is large enough that you’ll easily see such an effect in voting data if it was real. Does anyone think that in such an experiment we would see anything close to a .22% gain in voting propensity?

    (As an aside, at least in economics, Nature and Science are notorious for publishing “fun” results rather than cutting edge, carefully reviewed, research. I take their social science publications in general with a grain of salt…)

    Like

    afinetheorem

    September 13, 2012 at 8:18 pm

  4. they do know about voting. they looked up voter rolls. they note the differences between people clicking “i voted” and people who they could find records of their actually voting. the peer effects of clicking “i voted” are much stronger than the effects of actually voting. and the effects of casual friends exist for the report of voting but not for actual voting. for actual voting, only closest friends influences it. there is lots to buy or not buy in this. but first, you have to actually read the paper.

    Like

    Shamus Khan

    September 13, 2012 at 8:52 pm

  5. I can’t access the study here but can you describe how they “looked up voter rolls”? Are you implying that they have micro level voting data on 60+ million people and were able to successfully link those to Facebook profiles?

    Like

    Quetelet

    September 14, 2012 at 12:18 am

  6. […] Facebook field experiment shows strong ties affect voter turnout (orgtheory.wordpress.com) […]

    Like

  7. @Quetelet State voter files, which include vote history, are matters of public record. So yes, they could have gained looked up the voter rolls. Frankly, given the nature of the study, it would have been irresponsible of them not to.

    Like

    Bradley Spahn

    September 14, 2012 at 2:56 am

  8. People, please read the article before you criticize it. They did do a serious effort to measure actual voting, and the effect persists, although much smaller.

    @Brayden: indeed this study highlights the potential of online experiments, and this is also stated by the authors: “the growing availability of cheap and large-scale online social network data means that these experiments can be easily conducted in the field.” The problem is that such experiments are not easily conducted at all. Companies like Facebook own these type of data, and they only very rarely give researchers access. This particular study was only possible thanks to Facebook’s cooperation; two of the authors actually work at Facebook. Other researchers can not simply go and collect another 61 mln sample to replicate this study. That doesn’t mean that I think such studies should not be conducted, but the future of online experimentation may be more in home-made online environments like in Centola’s study (http://www.sciencemag.org/content/329/5996/1194.short).

    Like

    Rense

    September 14, 2012 at 11:14 am

  9. You can get a copy of the article from James Fowler’s website:

    Click to access massive_turnout.pdf

    The supplementary material, which is about five times as long as the article, is ungated on the Nature website:

    Click to access nature11421-s1.pdf

    Like

    Neal

    September 14, 2012 at 2:25 pm

  10. Brayden and Rense,
    Large companies increasingly do conduct a lot of experiments, as described in Jim Manzi’s Uncontrolled. The trick is whether they will share the results and/or consult in the design with academics pursuing theoretical interest. I’m thinking the best way to make this work is to go back to the Lazarsfeld “Bureau of Applied Social Research” model and do consulting work for companies in such a way that both benefits the firm and gives the researcher access to data / experimental interventions that can address theoretical concerns.

    Like

    gabrielrossman

    September 14, 2012 at 4:38 pm

  11. Did Shamus just argue that, with a population of 62 million, there will be an appreciable difference between a sample of 60 million and one of 6 million? Ouch.

    Like

    Moivre

    September 14, 2012 at 6:12 pm

  12. Moivre,
    Shamus was talking about the standard error and indeed, the standard error is about three times greater with 60 million than 6 million which is more than enough to bump a star off the table here and there. As for the sample vs population issue, I assumed Shamus was working with the superpopulation assumption.

    Like

    gabrielrossman

    September 14, 2012 at 8:15 pm

  13. excuse me I typed that wrong and meant 1/3, not 3 times. but you get the point, a ten-fold increase, even on a base of 6*10^6, does imply a 3:1 change in standard error and that’s nothing to sneeze at.

    Like

    gabrielrossman

    September 14, 2012 at 8:18 pm

  14. My understanding is what gabriel said… and if you look at the tables, I’m pretty sure it’s a difference that makes a difference.

    But I have a more important question: why are they running simulations to look at peer influence? So looking at the Supplementary Materials, on page 7 they say, “To take the network into account, we measure the empirical probability of observing a behaviour by a friend, conditional on a user’s treatment (see Figure S1 for examples of how the user treatment might cause changes in a friend’s or friend’s friend’s behaviour). A single user will be connected to many friends, so we conduct this analysis on a per-friend basis… To compare this observed value to what is possible due to chance, we keep the network topology fixed but randomly permute the assignment to treatment for each user and once again measure the per-friend treatment effect. We repeat this procedure 1,000 times. The simulated values generate a theoretical null distribution we would expect due to chance when there is no treatment effect. We then compare the observed value to the simulated null distribution to evaluate significance. We obtain confidence intervals for the null distribution by sorting the results and taking the appropriate percentiles (in our case, we are interested in the 95% confidence interval, so we use the 25th and 975th values). The random permutation method overcomes the problem of non-independent observations by taking the specific network structure into account when the null distribution is generated.” I get why they’re not doing OLS regression here, but still, it strikes me as a fairly elaborate technique where a far simpler analysis could have shown the effect of peer influence.

    As Moivre’s attempt at snark indicates, I’m an ethnographer not a quant person (I do enjoy the bravery of a nasty comment hidden behind anonymity). But it’s not clear to me at all why a far simpler comparison couldn’t have been done here.

    Like

    Shamus Khan

    September 14, 2012 at 9:31 pm

  15. I honestly don’t know who “Shamus” is and I have no stake in this research. I’m just an irregular visitor here, was struck by his statement, and doubly struck by the fact that nobody bothered to note that 6 million is a very large sample (to put it mildly). The notion that there will be some appreciable difference in point estimates between 6 million and 60 million samples from a 62 million population is simply incorrect. So, no attempt at snark, just a futile attempt to correct something wrong in the internet, in a corner of the internet where things aren’t as typically wrong as they are elsewhere. I now see that “not being a quant person” is insurance of a sort.

    Like

    Moivre

    September 16, 2012 at 5:12 pm

  16. […] Facebook field experiment shows strong ties affect voter turnout (orgtheory.wordpress.com) […]

    Like

  17. Moivre,

    Actually, it’s “SHAMUS” not “Shamus.” It’s an acronym standing for Simulated Human Anthropomorphic Machine User Simulation and serves as an artificial intelligence agent being tested in a real world field trial by posting to blog comments in the persona of a Columbia University sociology professor who does work on cultural capital. The SHAMUS AI is programmed with a very solid knowledge of Bourdieu and boarding schools ad a pretty solid but secondary understanding of standard error as a function of sample size.

    Like

    gabrielrossman

    September 16, 2012 at 10:38 pm

  18. @gabrielrossman: Got it. Thanks, that helps. How soon will they roll out FABIO? Sure, fully automated blog input-output algorithms still have a way to go, but I hear the CalTech team has FABIO producing some really clear prose.

    Like

    Moivre

    September 18, 2012 at 6:47 pm

  19. […] science 2.0: the already much-debated 61 million person experiment conducted on Facebook, as published in […]

    Like


Comments are closed.