orgtheory.net

the Q words

Regular orgtheory commenter Howard Aldrich has an interesting and provocative piece up at the OOW blog, Work in Progress, and the LSE Impact blog. His plea is that we should abandon the Q words — qualitative and quantitative — in describing our research. They aren’t terribly descriptive of what we’re actually doing, they create unnecessary divisions within social science, and using them inappropriately devalues qualitative work:

I’ve endured this distinction for so long that I had begun to take it for granted, a seemingly fixed property in the firmament of social science data collection and analysis strategies. However, I’ve never been happy with the distinction and about a decade ago, began challenging people who label themselves this way. I was puzzled by the responses I received, which often took on a remorseful tone, as if somehow researchers had to apologize for the methodological strategies they had chosen. To the extent that my perception is accurate, I believe their tone stems from the persistent way in which non-statistical approaches have been marginalized in many departments. However, it also seemed as though the people I talked with had accepted the evaluative nature of the distinction. As Lamont and Swidler might say, these researchers had bought into “methodological tribalism.”

Having recently argued that Sociological Science needs more “qualitative” work, I read this with interest. Certainly the terms are not the most descriptive, and they do reinforce a division within sociology that might better be blurred post-Methodenstreit.

But I think the distinction is likely to persist, despite Howard’s good intentions, for two reasons.

One is practical, or political. There is still a real status difference between the two broad categories of methods. Now, I’m not saying that qualitative research is totally marginalized, or that qualitative sociologists should walk around with a chip on their shoulder. But methods like interviewing, ethnography and fieldwork, content analysis, and archival research are seen as easier and less rigorous, perhaps because their squishiness is firmly on display, rather than being hidden behind numbers. Their deployers are often on guard against being squeezed out entirely, whether because universities are jumping on the latest research bandwagon (big data?) or demanding that faculty bring in external funding.

Under such circumstances, it is not surprising that people whose research practice is quite different tend to band together as a group. I basically do comparative-historical research. It’s very different from ethnography. But arguing that Soc Science should be sure to publish comparative-historical work is not going to bring much sympathy — it’s too narrow a critique. Besides, it is the broad tent of qualitative methods that is largely missing from the journal. I don’t see a way to raise such an issue without using the term “qualitative,” unless it’s simply by replacing it with a long list of specific methods, Ignoring it, on the other hand, may mean it goes unaddressed entirely.

The other is epistemological. The qualitative/quantitative distinction is often used as a placeholder for (and does overlap with) different epistemological positions, and different ideas about the purpose of social science. Daniel Little has discussed these at length, tackling the difference between qualitative and quantitative research, proposing multiple ways of distinguishing among methods, and a creating a five-way classification (below) of groups of social science methods.

blogger-image--61282874

And a comment on the Soc Science post by Bellerophon also highlights the epistemological side of the distinction. After quoting parts of the journal’s submission guidelines, Bellerophon writes:

Frankly, (and again said with admiration for some of the work that’s appeared in that journal), stuff like this reads like the wish-list of quantitative people’s frustrations after reading qualitative work. The substantive concerns aren’t so much the worry as is the odd positivist tone that privileges one kind of work over another, and rather than signalling “openness” to qualitative work, perhaps a better work is “toleration” for a certain kind of qualitative research.

At this point in the game, sociology has mostly agreed not to discuss its internal epistemological differences so that we can all get along. Most quantitative (there’s that word again) researchers really do think that sociology needs ethnography and so on, and most ethnographers acknowledge there’s value in fixed-effects models, even if both types might prefer a bit more of their own methods and a bit less of the other sort. No one wants to waste (more) time bickering about something that’s unlikely to change anyone’s mind. And to the extent that that’s what Howard is asking for, I’m with him.

But sometimes arguing about these distinctions can actually be good for disciplines. I think King, Keohane and Verba’s Designing Social Inquiry: Scientific Inference in Qualitative Research was deeply misguided in its prescriptions for good research. But it generated a conversation in political science (and comparative-historical sociology) that greatly improved the state of comparative and case-based methods. Now, whether you can find those methods in APSR or AJPS is a different question. But the state of knowledge itself has advanced.

So I understand Howard’s frustration, and I’ll certainly be more attentive to how I use the Q words in the future. There are definitely times when more precision is called for, and it’s not a good idea to pretend that what is actually an epistemological debate is a methodological one. But despite all that, I think there are still times when the distinction is called for. I’m not ready to give it up entirely.

Advertisements

Written by epopp

November 28, 2014 at 5:58 pm

10 Responses

Subscribe to comments with RSS.

  1. Your (Little’s?) pie chart appears to lump everything that uses numbers together as survey research. And appears to embody the assumption that everything that uses numbers has the same epistemology.

    Like

    olderwoman

    November 28, 2014 at 8:01 pm

  2. Yes, that’s Little’s chart, not mine, and I agree that it doesn’t break things down in a completely satisfactory way. I originally had something in there about formal models and RCTs being quite different from each other, but the post was getting long.

    Maybe the bigger point is that there are many epistemologies (at least more than five) underlying the various methods.

    Like

    epopp

    November 28, 2014 at 8:04 pm

  3. I notice Howard still lumps together “non-statistical.”

    Like

    Philip N. Cohen

    November 29, 2014 at 11:45 am

  4. I think that research counts as “quantitative” if the data was made using a procedure that is meant to allow you to make assumptions about the error distribution for the sake of analysis. At least what’s they usually mean by it.

    Like

    Unlearner

    December 1, 2014 at 12:11 am

  5. I’ll believe Little’s typology when I see it established through Latent Class Analysis of a dataset with a well justified sampling frame.

    Liked by 1 person

    gabrielrossman

    December 1, 2014 at 10:37 pm

  6. […] Note: Interested readers should check out Elizabeth Popp Berman’s response to Howard’s post, which she posted on […]

    Like

  7. @gabrielrossman Wouldn’t you also demand that it be replicated cross-nationally? On multiple representative populations? Maybe use SEM? Stop me before I make a complete fool of myself…
    @familyunequal [aka Philip Cohen] Nice use of Ngram! When I was discussing the issue of labeling types of research with my friend, Diane Vaughan, she told me that no one ever introduced her/himself to her as “I do quantitative research.” The very phrase sounds silly, doesn’t it?
    @Unlearner I’ve never seen a consensus definition of “quantitative” along the lines you describe. I can see how some people, in some cases, might say that…But as Philip says, my point is about the negative framing involved, not getting the definition right.

    Like

    Howard Aldrich

    December 2, 2014 at 6:22 pm

  8. One issue with qualitative research is that, at the very least, researchers are measuring whether a non-trivial number of subjects said something. As a result, you’re setting a quantitative benchmark. To set this benchmark well, you should know how to compute statistical power, which entails quantitative expertise. To quote one of my favorite conference presentations:

    What can’t you reliably detect with n = 20?

    People who like spicy food are more likely to like Indian food (n = 26)
    Liberals rate social equality as more important than do conservatives (n = 34)
    Men weigh more than women (n = 46)
    People who like eggs report eating egg salad more often (n = 48)
    Smokers think smoking is less likely to kill someone than do non-smokers (n = 144)

    Like

    chrismartin76

    December 3, 2014 at 2:18 am

  9. […] piece arguing to abolish the distinction, and thereby reinvigorated the debate, for example on OrgTheory and Work in Progress.  A quick look at Google’s Ngram viewer is quite interesting. For […]

    Like


Comments are closed.

%d bloggers like this: