a crazy proposal: universal peer review

I had a crazy idea while talking to some friends over dinner Saturday night. Why not create a universal peer review system that applies to all journals in a field? The problem with the peer review system, as many see it, is that it takes so darn long to get a paper from first submission to publication stage and it tends to exhaust really good reviewers (who sometimes may be asked to review 3-4 papers a month). The world would be a better place if we could kill these two birds with one stone by shortening the time to publication and using reviewers’ time more efficiently. A universal peer review system would deal with those problems.

The idea stems from the law review system in which people submit their papers to a clearinghouse where it can then be assessed by all of the law review editorial staffs that the author targets for submission. You can submit to as many law reviews as you’d like. Once one law review accepts the paper  a bidding war begins between the reviews. The author almost always takes the offer from the most prestigious law review, which often doesn’t come until a number of lower-tier publications have made their case for why you should publish the paper in their review. It may be easier to convince a higher tier journal to accept your work if you already have 15 acceptances. The system is fast. You submit papers twice a year, which is followed by 2-3 months of negotiation until all of the publishable papers are divvied up among the reviews.

Someone suggested it would be great to have the same process with management (or sociology) journals, but the barrier, of course, is that we rely upon peer review, whereas law review editors rely upon their own intrinsic criteria.  You could make it work though, I think, if you were to create a universal peer review system. Authors would submit their work to a clearinghouse and check the journals to which they wanted to submit. The system would then pick reviewers from an approved list of reviewers from the set of journals that the author submitted to. You would probably pick more reviewers (4 or 5) given the need to ensure an accurate estimation of quality. In addition to the normal feedback and ranking of quality,  reviewers would be asked to rate the journal in terms of its appeal to various audiences (e.g., does this paper appeal to a general management audience? sociological? economics? etc.) The reviewers themselves could be ranked according to reliability and taste so that the editors could assess the fit of the paper to their journal. Once all of the papers are reviewed (you should be able to do this in a month since reviewers will only be doing this twice a year), the editors could start sorting the database and finding papers that fit their journal’s criteria of quality. Once they’ve identified some subset of papers they find worthy of publication the bidding could begin. Once a paper was accepted, of course, the editor and perhaps a subset of the first-round reviewers would commit to work with the author until the paper was in acceptable shape for publication; however, the biggest chunk of work for reviewers would come at the front end, when assessing the quality of papers.

Why would this work? Well, for obvious reasons it would be great to have a system that pushed papers to publication quicker. Many papers, especially those that deal with timely topical issues, lose their relevance unless published immediately. Even theoretical papers are sometimes outdated by the time they are published. I see this as a huge problem with our field. It would also put more incentive on the authors to publicly air and get feedback on their papers before submitting their work to journals. While I’m okay with a paper being submitted that isn’t 100% ready (what paper is?), I’m afraid that too many low quality papers are submitted simply because the author is looking for developmental feedback. With a one-and-out system like this, authors would be more diligent in seeking feedback and polishing their work before submitting. I also like the idea because it would allow innovative ideas to rise to the top. During the bidding process, naturally intriguing papers that might not fit the conventional model of a journal might suddenly appear more publishable if they are vigorously pursued by many other journals in a field.  Perhaps the best reason for doing this is that it would vastly cut down on the amount of time scholars spend reviewing work. Some papers get submitted 5+ times before being accepted. Multiply that by the number of reviewer hours you used to move that paper to publication and you get a sense of the inefficiencies of the current system. (I’d argue that most papers are only marginally improved by going through multiple rounds of review.)

I’m sure that you readers can think of many reasons why this system wouldn’t work, but I’m willing to bet that we could collectively figure out a way to implement the system despite the reservations. It may seem like a crazy idea, but I think it’s a potentially very effective solution to our field’s inefficiencies.

For more crazy ideas on improving the peer review system, see Fabio’s suggestion for a triple-blind peer review process.