what is “bad” research?

Last week, we had a fruitful discussion of graduate school and publishing. I think we all agreed that most graduate students should learn how to publish quickly. But we also raised some red flags. For example, we shouldn’t encourage people to publish “bad” articles. Others thought that we shouldn’t publish “too much.”

So let’s begin with a consensus: yes, if you are a graduate student, you should definitely learn the publishing process. No let’s move on to lower consensus issues. First, what counts as “bad” research? A few definitions:

1. Research that is fraudulent.
2. Research that is in a technical sense correct, but misleading.
3. Research that is sloppy or poorly written.
4. Research that is made in good faith, but in error.
5. Research that is chopped up into lots of small chunks, in terms of article length/word or page counts.
6. Research that makes extremely small or incremental arguments.
7. Research that is in the “wrong” journal – low prestige, niche, online, or in a lower status discipline.

Now, when is it bad to publish work in any of these categories? There is overwhelming consensus that #1 is bad and should never be tolerated. In fact, academia has such a strong norm on #1 that fraudulent articles are almost always retracted and people might lose their job. I think we’d agree that #2 is also bad, though there is disagreement about what should be done with misleading articles, as we found out when discussing He-Who-Shall-Not-Be-Named-in-Texas.

Once we get past fraudulent and misleading research, it’s very unclear that any of the remaining categories can be claimed to be uniformly bad. For example, garbage can paper (March, Cohen, and Olson 1972) was successfully shown to be a very sloppy work (see the Bendor, Moe, & Schott 2000 APSR article). No way around it. But, as olderwoman points out, powerful ideas are often presented in sloppy packages.

Then we get to #4: good faith papers with mistakes. In some cases, #4 is obviously bad. We find out that the answer is different when we correct our code – retraction. But in other cases, it’s ok. For example, among mathematicians, incorrect proofs are sometimes left in the record. The overall idea remains promising, but maybe some future scholar can read the mistake and fix it.

#5, #6 and #7 are clearly not universal. If you look around academia, you see that some fields hate, hate, hate small articles (history) while other fields exist primarily in tiny, tiny articles spread out in big and small journals. Even with one field, like sociology, you see huge variance. Demographers routinely “chop and spread it,” while ethnographers save it all for one big AJS/ASR article.

I’ll finish with how I think about my own publication strategy. My first allegiance is to knowledge. So I have never suppressed any article that I thought had a specific contribution to make, big or small. Also, in my own experience, I have benefited greatly from articles published in some obscure places. “Small” doesn’t mean dumb or useless. Just small – which might be very important to someone out there (including me).

What I have ended up with is a sort of triage: articles that are “big” in some sense are channeled to major journals, while “small” contributions are sent to niche journals. That results in a output stream where the modal is “small” but the stream is punctuated by a few “bigs.” Finally, one thing that I don’t do is rewrite the same article over and over. I make no claim that this is optimal, only that this is what you get if you believe that “small” contributions and niche journals have a place in the academic world.

Adverts: From Black Power/Grad Skool Rulz

Advertisement