comments on andrew gelman’s dec 21 post

Last Saturday, Andrew Gelman responded to a post about a discussion in my social network analysis course. In that post, my student asked about different strengths of a network effect reported in a paper. Gelman (and Cosima Shalizi) both noted that the paper does not show a statistically significant difference. I quote the concluding paragraphs of Andrew’s commentary:

I’m doing this all not to rag on Rojas, who, after all, did nothing more than repeat an interesting conversation he had with a curious student. This is just a good opportunity to bring up an issue that occurs a lot in social science: lots of theorizing to explain natural fluctuations that occur in a random sample. (For some infamous examples, see here and here.) The point here is not that some anonymous student made a mistake but rather that this is a mistake that gets made by researchers, journalists, and the general public all the time.

I have no problem with speculation and theory. Just remember that if, as is here, the data are equivocal, that it would be just as valuable to give explanations that go in the opposite direction. The data here are completely consistent with the alternative hypothesis that people follow their spouses more than their friends when it comes to obesity.

Fair enough. Let me add a pedagogical perspective. When I teach network science to undergrads, I generally have a few goals. First, I want to show them how to convert social tie data into a matrix that can be analyzed. Second, I want students to learn how network concepts might operationalize social science concepts (e.g., how group cohesion might be described as high density).  Third, I want to spark their imagination a little and see how network analysis can be used to describe or analyze a wide range of phenomena and thus encourage students to generate explanations. Given that students have very, very modest math skills and real problems generating hypotheses,  getting down into the weeds with the papers is often last.

So when I teach the week on networks and health, my discussion questions are like this: “Why do you think health might be transmitted from one person to another? How would that work?” I also try to get into basic research design: “How do you measure health? Do you know what BMI is?” So the C&F paper has many up sides. The downside is that the paper has an interesting hypotheses and you can easily get distracted from the methodological controversy the paper has generated, or even some very sensible observations on confidence intervals. The bottom line is that when you have to teach everything (theory, methods, research design and topic), you don’t quite get everything. But still, if a student, who self-admitedly knows little math or stats, can get to a point about asking about mechanisms, then that’s a teaching victory.

Post-Christmas blow out: From Black Power/Grad Skool Rulz