orgtheory.net

a simple way to speed up the journal review process

One of the main problems of scholarly publishing is that it takes too long. One reason is that there is simply no competition for articles. Since journals have a “single submission” policy, you are giving each journal a monopoly right to your publishing until they have the time to reject your paper. Not surprisingly, journals treat you like garbage. Personally, I once had an editor lose a paper twice (!), papers rejected after editors appoint entirely new reviewers, endless demands for revision from editors who won’t structure the revision process, and so forth. I can  take rejections, but don’t string me around. You are wasting my time if there isn’t a decent chance you will take my paper.

I think there is one simple way to improve the journal process: abolish the “single submission” rule. In other words, submit the paper  to as many journals you want at the same time until you get an accept. If multiple journals offer to publish, then you choose where it goes.

A while back I suggested this and people responded in a negative manner. But consider this:

  • You can submit applications to as many colleges as you want.
  • You can submit applications to as many grad programs you want.
  • You can submit applications for as many jobs as you want.
  • You can submit funding applications to as many agencies as you want.
  • You can submit book manuscripts to as many publishers as you want.

And they all work. The system does not break down. In all cases, you simply negotiate multiple offers if they happen and choose one.

The only reasonable objection is that reviewers may be overloaded. That is a response with a simple answer. If a reviewer is asked to review a paper more than once, they either decline the review (clicking a link in an email) or clip and paste the old review. If you are worried that journal editors will be overloaded, you can add some simple restrictions to lighten the load. For example, authors can only submit one paper every six months. Or if a paper is accepted, you can’t submit for another six months. Perhaps the simplest solution is a submission fee, which is used in many fields. In other words, reviewer overload is easy to solve. Journal overload is also quite solvable. There is no reason to maintain single submission and its abolition will quickly improve academic journal publishing.

50+ chapters of grad skool advice goodness: Grad Skool Rulz ($2!!!!)/From Black Power/Party in the Street

Written by fabiorojas

February 24, 2016 at 12:01 am

29 Responses

Subscribe to comments with RSS.

  1. One of the subfields in Management Information Systems is experimenting with a “paper marketplace” to try to solve the same problem.

    The ultimate idea is that journals can “bid” on papers that they think will be valuable, based on assessments by experts in the field. More information is available at the market’s website: http://journal-marketplace.org/the-process/

    Like

    kjetha

    February 24, 2016 at 12:33 am

  2. See: law reviews. Of course, there are some substantial differences in the models (student editors, submissions aren’t blind).

    Like

    Mikaila

    February 24, 2016 at 1:00 am

  3. “submission fee” ??? We have submission fees.

    Like

    olderwoman

    February 24, 2016 at 1:16 am

  4. Abolish journals. Publish first, review after. ASR can spend as long as they want deciding which already-available papers they want to bless with their high-status holy water. Meanwhile, we can all read the work.

    Liked by 3 people

    Philip N. Cohen

    February 24, 2016 at 1:36 am

  5. I think that this is a really good proposal. The one submission rule is a clear opportunity hoarding device that works against the interests of authors and reviewers. For book prospectuses, as well as job applications, we already submit multiple applications, why not for journal submissions?

    Liked by 1 person

    Donald Tomaskovic-Devey

    February 24, 2016 at 1:59 am

  6. The real problem with the proposal is not a situation when one reviewer receiving the paper from multiple journals, it’s the situation when different reviewers get the same manuscript.

    Assume a paper is good enough to be published after the first round of reviews. Under the current system, three reviewers put time and effort into reviewing the paper, and it’s accepted. (Or 2+1, or whatever — same principle holds with multiple rounds of review.) Under your proposal, if the author sends it out to 4 journals, 12 people spend their time on the paper; 9 of these sets of comments end up being completely ignored. How, again, is this supposed to be collectively efficient?

    I’d guess what would happen is that editors would have to ask 4 times more reviewers before getting 3 acceptances, a greater share of people will just start saying no to every request (I often have days when I receive 2 or 3 requests — multiply that by 4, and I’m probably going to set my filters so requests go straight to spam), and the system bogs down under the collective weight of an unbelievable amount of wasted effort.

    Like

    krippendorf

    February 24, 2016 at 2:59 am

  7. Krippendorf assumes that the review process does not adjust. Currently we use 3 reviewers, ad 2 or 3 at the R&R stage, both so that the editor does not have to deal with ties or make decisions. I would advocate two reviews and an editor’s up, down, or R&R decision. In the case of R&Rs the editor decides, no second rounds. I would be this would reduce reviewer burden, or at least redistribute it but not increase it.

    Liked by 1 person

    Donald Tomaskovic-Devey

    February 24, 2016 at 3:09 am

  8. OW: Sorry, I still submit to venues that do not have fees.

    DTD and Krippendorf: I side with DTD. No reason to assume rigid practice. People will adjust. But let’s be realistic. Few of us write papers that usually get accepted after one round. Most of my papers (and those of most academics) tend to bounce around a lot. My proposal merely compresses the time.

    And honestly, do you not realize that books get published all the time with multiple submission? And law reviews? There is no evidence that single submission makes a difference.

    Like

    fabiorojas

    February 24, 2016 at 4:04 am

  9. The journal system will die. But don’t let me stop you from devoting yourselves to rearranging the reviewer’s chairs.

    Liked by 1 person

    Philip N. Cohen

    February 24, 2016 at 4:21 am

  10. Put another way – why create a system of multiple submissions instead of fewer journals? You really think things would be better for everyone if you could submit your paper to Soc Inquiry, Soc Forum, Soc Perspectives, Soc Quarterly and Sociology all at once? I’m sorry but that’s nuts. Publish the paper. Let whoever wants to review it review it. While people – you know, people! – read it. Shocking, I know, but it is one way to think of the publishing process.

    Liked by 1 person

    Philip N. Cohen

    February 24, 2016 at 4:26 am

  11. Phil,I actually agree with you and I’ve put two or three papers in SSRN and arXiv. But right now, the soc discipline is not ready for what you propose. Single submission is a step that people can take. It won’t improve reviewing quality, but it will at least speed things up.

    Liked by 1 person

    fabiorojas

    February 24, 2016 at 4:29 am

  12. A middle-of-the-road solution, that avoids most of the problems of multiple submissions and single submissions, is to have a cap of having the article in two places at the same time max. That way, there is still the issue of competition and journals will have to hurry up if they don’t want the other journal to get the article first.

    Like

    sebastianguzman

    February 24, 2016 at 4:58 am

  13. Many fields of science already work on a modified version of this model, with two tiers of journals: open submission and “right of first refusal” journals (generally, the higher-status journals, such as Science and Nature, fall the second category). The latter require you to give the journal priority in publishing. You can only submit to one “right of first refusal” journal but you can simultaneously submit to multiple open submission journals. If one of the open submission journals offers to accept your paper, you tell the editor of the “right of first refusal” journal and s/he has a limited period of time to exercise the option to publish (a week or two).

    AJS and ASR could maintain much of their status by adopting a “right of first refusal” policy but that would allow the review process to speed up considerably. FYI, Sociological Science does not prohibit simultaneous submission at other journals.

    Liked by 3 people

    Olav

    February 24, 2016 at 10:01 am

  14. I wish reviews were portable. Sociologists regularly receive rejections from generalist journals despite generally positive reviews (“Though the reviewers find a lot to like about your paper, we are concerned it is not of broad enough interest to the discipline…”).

    I’d love the option to forward my paper + reviews (“Paper is methodologically sound but is a better fit for a field journal”) directly to a field journal for consideration. Editors would of course have the right to refuse a paper + review package should they have doubts about the reviewers. Or they could just read the forwarded reviews and make an immediate decision without needing to bother new reviewers. If this became common, it would reduce reviewer load and publication times.

    Liked by 3 people

    Carl

    February 24, 2016 at 3:20 pm

  15. It seems like a lot of this could be solved by giving editors more leeway to reject papers before sending them out for review. To Carl’s point, if the paper doesn’t have a broad enough appeal then that should be obvious to the editor on the first read through. The scope of the article won’t change in review and, based on my experience, tends to narrow as the paper progresses rather than broaden.

    Sending it out for review just to have it rejected now a) slows the progression of science, b) wastes reviewer’s time because, by the editor’s own admission, they didn’t think the paper would fit their audience, c) wastes the author’s time, and d) wastes the editor’s time and goodwill finding and receiving reviews.

    I am not sure that a stronger “desk reject” would work in our discipline because demographers would cry foul that their papers weren’t sent out, or ethnographers, or urban sociologists, or gender researchers, or sociologists who use comparative historical methods, or [fill in the blank].

    On another matter, it strikes me as odd that Phil Cohen advocates getting rid of peer review given how much effort he has devoted to getting a certain article retracted. If his idea goes though, that paper goes out into the public as science and gets the same weight as any other paper on the topic in court briefs, public opinion, etc. Somehow we imagine that people — you know people — have the same interest in science that we scientists do.

    Like

    mike_bader

    February 24, 2016 at 3:38 pm

  16. That is a good point, mike_bader. Although, FYI, I haven’t actually devoted much effort to getting that paper retracted – more to debunking it. There is no stopping papers from existing.

    I think the post-publication review could be organized to provide a more effective seal of approval system to fulfill that role you’re talking about. You could easily establish a review committee, with an editor, that reviewed and rated papers submitted to a common publishing platform – very efficient, no duplicative reviewing, no rejection for “fit,” etc. Then the paper would look bad in court if it had, say, zero stars from post-pub review committee.

    Anyway, I don’t want to hijack the thread with that. On the point of reviewer portability that Carl makes, I like that – and why not do that right now?

    “Dear Journal of Minor General Importance: I’m not going to lie and pretend this paper was written just for your journal. Yes, I did wait 3 months to get it rejected by a higher-ranked journal, and you are my second choice. Because people have already invested time in reviewing it, and I haven’t really changed it, I am providing you with copies of the reviews that I received, which may help you in making your decision.” (Or, “I have revised it to meet the suggestions I got from the High Status Reviewer Pool.”)

    Liked by 1 person

    Philip N. Cohen

    February 24, 2016 at 4:07 pm

  17. Phil Says:

    “You could easily establish a review committee, with an editor, that reviewed and rated papers submitted to a common publishing platform – very efficient, no duplicative reviewing, no rejection for “fit,” etc.”

    This is actually pretty similar to the Plos One publishing model. Something that seems to have taken off a lot more in the natural sciences than the social sciences

    Like

    JD

    February 24, 2016 at 6:14 pm

  18. I believe the real problem here is the time that a single round of review takes. In the real hardcore sciences, you rarely heard of review time more than six months. Some ground-breaking work took even less than a month (the twin prime number conjecture) to get accepted. Yeah, all the story-line junk. Are we writing novels/poems or scientific papers? I just think that we need to change the culture.

    Like

    Joe

    February 24, 2016 at 8:16 pm

  19. So only the lower-tier journals take the risk. I wonder if with the two-tier model, authors prefer to submit their best work only to the top-tier, worrying that they would have to accept something in the lower tier jut because they are faster. If top-tier journals think this will happen, they have less of an incentive to rush because they know that the best work is not being submitted to both tiers, especially if they think those works will need some revisions to fit their journal but that a lower-tier journal will accept them in their current state. This problem would be avoided if you can submit to two top-tier journals at the same time. Of course, you can’t impose a review system on journals and high status journals may not want to change. Unless the society behind them wants to. But also, none of this works if only one journal, like Sociological Science, accepts multiple submissions.

    So a practical proposal would be that if several of us here agree with Fabio’s idea, the proposal be submitted to the ASA’s committee on publications (and the SSSP and regional societies’ as well). If ASR and other ASA journals do this, they will effectively force other journals to do so, because authors will begin to submit more to the journals that accept multiple submissions, leaving them only with works that could not fit two multiple-submissions journals or which have been rejected from them already.

    Like

    sebastianguzman

    February 24, 2016 at 9:36 pm

  20. Sebastian: I was thinking exactly this. If ASA move first and abolished single submission for all ASA journals, then they can probably pull SF, SP and the AJS. Then, you can get the regionals. And that is probably enough.

    Like

    fabiorojas

    February 24, 2016 at 9:40 pm

  21. As a graduate student, it is really very gratifying to hear senior scholars collectively think of ways to improve the publishing process – our bread and butter – and even take concrete steps to make these ideas into actionable plans (e.g. submitting a proposal to ASA committee on publications). I believe I can speak for all aspiring sociologists when I say thank you.

    Liked by 1 person

    GradStudent

    February 25, 2016 at 3:21 pm

  22. I like fabio’s proposal, but as an occasional reviewer, I would have reservation reviewing papers that possibly won’t get published at the journal that I review for simply because of this multiple-bidder scenario. But I also get sick of the sometime ridiculously long review process and/or excessive number of review rounds. Here is my proposal:

    1. set up time frame constraint: during the first two months, submissions should be bound with the journal. After that, authors are allowed to send their papers to another journal if no results are returned within the two-month time frame. Once papers are in a multiple bidder scenario, the authors need to notify all editorial offices (I am not sure on this).

    2. set up a fair review coalition: scholars in this coalition will not accept a third R&R or a review process that takes more than six months, and this needs to be made clear during their initial submission. Since the cost is high, only senior scholars or scholars who do not have serious concerns by so doing are encouraged to join this coalition. This coalition should ally with some open source journals, such as socius or sociological science. Paper rejected in their third round else where then have outlet at these journals with an expedited review.

    I am just tossing these out for better ideas….

    Like

    Joe

    February 25, 2016 at 11:01 pm

  23. As a comparison, the review process in management has sped up dramatically over the last 5-7 years. It used to be that 3-4 months was the “expected” time from submission to decision (in each round) and 6 months or more was very common. Now most management journals ask reviewers to get their reviews back in 4-6 weeks (depending on the journal) and most reviewers do. The purgatory of waiting for a journal decision is a whole lot less painful when it’s 2 months than when it’s 6, 8, 12 months or longer. Especially for junior faculty.

    Ultimately, I agree that it is an issue of culture. Most of us spend no more than 2 days actually doing the review once you sit down and start — the issue is whether you wait 3 weeks or 3 months before starting and management has been largely successful at promoting a cultural norm of getting reviews done quickly that makes the whole system work better. Not perfect, by any means, but better.

    Like

    akleinb

    February 26, 2016 at 1:25 pm

  24. @akleinb I agree. It’s a matter of changing the culture. It’s ironic that sociology, a discipline that is supposed to be ahead of others in coming up with the fairest system, is way behind. That probably speaks to the status of our discipline among all.

    Like

    Joe

    February 26, 2016 at 3:57 pm

  25. The problem with getting rid of developmental reviews, which is what is being advocated, is that as senior scholars, you’ve benefited from these developmental reviews early in your career – it’s what has taught you how to write/frame a strong paper.

    Like

    Sophia

    February 27, 2016 at 1:53 pm

  26. very good point

    Like

    Donald Tomaskovic-Devey

    February 27, 2016 at 2:29 pm

  27. I think it might be worthwhile to try and save the system broadly as is by just speeding it up significantly. This means that clear deadlines should be set and adhered to. And this shouldn’t only involve reviewers and editors, but also those writing and submitting the papers. I’ve experienced many times that it’s often the scholars themselves who are to blame for excessively long times before a paper gets finally accepted.

    So, give editors two weeks to decide if desk reject or send out; reviewers two months to perform peer review; and the editor again two weeks to make the editorial decision–three months turnaround time in total. Then two months for revising and resubmitting the paper. If we kill the 3rd round R&R, and instead make decisions after the second round, we should collectively do a better job in producing novel ideas and create new knowledge, as opposed to research that’s been around for many years already, but seen only by editors and reviewers.

    Not meeting these deadlines should involve some penalties. Scholars, for example, could be banned to resubmit to that journal for a year or two. And editors and reviewers may just be negatively evaluated, reducing their chances of keeping their respective roles, or advance to other functions in boards of other journals in the field etc. Having said that, I agree that much of this will need to involve more of a cultural change, as opposed to strict coercive pressures.

    Of course, there need to be room for exceptions, for example in cases of sick or parental leave, or other special circumstances. But it should be understood that these are really exceptions, rather than the norm.

    Like

    Guest Comment

    February 29, 2016 at 12:52 am

  28. I’m with you on most of this, but not the idea that you penalize an author for being unable to turn around a manuscript in two months. It’s fine if the revisions requested are minor–add a couple of citations, explain a given methodological choice, etc. But when revisions requiring major re-writing, running of new models, or other intensive work, two months is not necessarily sufficient. Remember, many of us has working lives which do not provide time for sustained engagement in our research during the semester. When we are asked to review a paper, we can take our schedules into consideration when making the decision as to whether to say yes or no. But if we submit a paper in June and get back a lovely, speedy R&R in September asking us to resubmit by Thanksgiving or else face a publishing ban, all while teaching 3 or 4 classes with no research or teaching assistance, what are we supposed to do?

    Liked by 2 people

    Mikaila

    February 29, 2016 at 12:59 am

  29. […] few weeks ago, I suggested that we could speed up reviews by having “multiple submission” in sociology. Submit to as many journals as you want, when you want. Here is a draft of a petition for the […]

    Like


Comments are closed.

%d bloggers like this: