should sociologists stop interviewing people?

My friend Colin Jerolmack and scatterista Shamus Khan have a new article in Sociological Methods and Research that criticizes the way many social scientists use interview data. From “Talk is Cheap:”

This article examines the methodological implications of the fact that what people say is often a poor predictor of what they do. We argue that many interview and survey researchers routinely conflate self reports with behavior and assume a consistency between attitudes and action. We call this erroneous inference of situated behavior from verbal accounts the attitudinal fallacy. Though interviewing and ethnography are often lumped together as ‘‘qualitative methods,’’ by juxtaposing studies of ‘‘culture in action’’ based on verbal accounts with ethnographic investigations, we show that the latter routinely attempts to explain the ‘‘attitude–behavior problem’’ while the former regularly ignores it. Because
meaning and action are collectively negotiated and context-dependent, we contend that self-reports of attitudes and behaviors are of limited value in explaining what people actually do because they are overly individualistic and abstracted from lived experience.

Overall, I find much to like in the article, but I wouldn’t get carried away. First, interviews and surveys vary in the degree of bias. I probably trust a question about educational history more than I do, say, racial attitudes. On a related point, you can also assess the quality of questions. Political scientists have definitely found biases in survey questions and that tells you how good a question is. Second, in some cases, you don’t have any choice but to work with interviews and surveys. For example, interviews are crucial for historical work. So when I do interview research, I use with caution.

50+ chapters of grad skool advice goodness: From Black Power/Grad Skool Rulz