orgtheory.net

Brayden

Tyler Cowen and Henry Farrell are having an interesting discussion about opening up the academic publication process. Tyler thinks that we’d benefit from having “a free access system” in which authors could post their papers online, papers could be ranked by readership and citations, and paper quality could be assessed by these metrics rather than by acceptance to a prestigious journal. Tyler also thinks this would reduce the problems associated with getting good reviewers. Since there are few incentives for reviewing anyway, we could simply bypass reviewing and allow the attention market to decide which papers were worth something. Henry is skeptical that there is a reviewer problem to worry about. People don’t review papers because they have incentives, he thinks; they review because it is normatively appropriate to do so.

As it turns out, Linda Trevino, professor at Penn State’s Smeal College of Business and an associate editor at the Academy of Management Review, wrote about this topic in January’s issue of AMR. From the editor’s comment: “Why Review? Because Reviewing is a Professional Responsibility”:

[S]cholarly journals simply cannot operate effectively without a strong peer review system, which has been central to science for more than 300 years (Weller, 2001). Certainly, the internet has made it possible to post working papers so that others may comment on them. Some of you may believe
that such a system is preferable because it speeds information into the marketplace of ideas and opens up what many perceive to be a closed system. In such a system, anyone can publish anything (Laine & Mulrow, 2003). Put it out there and let the marketplace decide. This may sound good in theory, but it fails in practice. I don’t know about you, but I rarely read working papers posted on the web. I simply don’t have time. I count on quality journals to sift through the huge amounts of information available and to signal what is most worth reading, worth assigning as required reading in doctoral seminars, worth taking into account in my own research, and worth translating for students and practitioners in my textbook. Although we all know that top-tier journals make mistakes, recent research suggests that many of you agree with me, because, in your own research, you are more likely to cite work that has been published
in the field’s top-tier journals (pg. 9).

Trevino suggests, correctly I think, that journals serve an important function independent from housing articles. Journals signal which papers are worth reading. Sure, you could imagine that other signals might operate in a free access system, but they would likely be tethered with additional sorting problems: information cascades, author prestige would matter more, etc. How would a new scholar, especially one who is at a lower-ranked school, get discovered in such a system? It seems like the costs for searching for high quality papers and for breaking into the system as a new scholar would be much higher in a free access system.

Trevino also addresses the issue of reviewer incentives. She notes that one of the main problems confronting any journal editor is finding good reviewers. A surprising number of scholars simply refuse to review. Like most of us, she thinks reviewing is a professional responsibility. Ethically, you should feel compelled to review papers at journals that you read and to which you submit papers. Unfortunately, a number of people refuse to review (keep in mind that she’s talking about seeking reviewers for AMR, one of the top journals in organizational scholarship; the headaches that editors at lower-ranked journals have are probably much greater).

She notes that there are many good reasons for choosing not to review a paper. You may not read the journal or ever seek to publish in the journal, you may be too closely associated with one of the authors, or the paper may lie outside your area of expertise. These are all good reasons for not reviewing. But some scholars simply don’t review, period. The evidence, then, suggests that, as Tyler thinks, there is at least a minimal reviewer incentive problem. Her solution?

As ethics ombudsperson for the Academy (another Academy role I play), it occurred to me that perhaps we should consider adding a statement of this responsibility to our ethics code and enforce it. It’s fine for someone to “opt out” of reviewing. But, if so, shouldn’t that individual “opt out” of the submission process too? The Academy journals now have a very efficient web-based tracking system, so these individuals are easily identifiable. What do you think? Should we consider enforcing the quid pro quo expectation in some way?

If scholars are going to treat the journal in an instrumental way, refusing to review because they don’t want to bear any of the collective costs of producing a quality journal, shouldn’t the journal have the right to refuse their article submissions as well? Trevino’s suggestion is an interesting one. Notably, she’s taking Tyler’s logic to the extreme. Normative constraints might not be enough; professional responsibilities/ethics may have to be actively enforced.