orgtheory.net

let’s talk about third tier journals

My question: how should academia approach small, third tier journals?* In a previous post, I linked to research showing that these journals do actually publish some highly respected material. The gist is that the average third tier article isn’t cited much, but these journals publish occasional winners who do much better than the bottom articles in the elite journals. The issue is that when it comes to rewarding people, we have essentially two models: (a) immediate reward for publication in top journals/presses or (b) you wait a few decades when it’s clear that a low status publication was actually any good. Is this a good model?

There’s a standard justification for this policy. Publication in a third tier journal is usually seen as a clear signal that the research is weak. Basically, “third tier publication” = failed research article. A few comments on the policy: First, a journal may be third tier simply because it has a narrow audience, not due to quality. Second, top journals may make mistakes and reject good articles. Reviewers may not like a new argument, or the topic may seem odd. Third, statistically speaking, many third tier articles aren’t good, so I believe the policy has some justification.

Now, if you completely believe in the standard rap against third tier journals, then there are some serious implications. One is that all the tiny journals should be shut down immediately. No need to waste time or resources on junk. Another implication is that you should count these articles against people. Don’t reward people for junk. If you are skeptic, then you should be at least agnostic. You shouldn’t be terribly impressed with third tier articles, but you might adopt a more agnostic stance toward their quality.

* I define “third tier” to be legitimate academic journals that aren’t flagship journals, well known speciality journals, or those published by major national or regional associations. These include most student run journals, journals run by obscure departments or occupational groups,  journals for unpopular specialties, journals from small countries, or journals that have simply not risen to the top in a well regarded specialty.

Written by fabiorojas

January 26, 2009 at 2:09 am

Posted in academia, fabio

23 Responses

Subscribe to comments with RSS.

  1. These arguments make perfect sense, historically. However, as expectations for publication rise higher and higher, new issues arise. It used to be that graduate students could expect to get good jobs without having published peer reviewed work. It used to be that community college faculty and lower-tier teaching-focused institutions’ faculty could earn tenure without publication. It used to be that several good articles were enough at research institutions, rather than there being demand for continuing production of quantity. Now, there is simply so much output that the first and second tier journals cannot contain it any more. Things that would have been acceptable several decades ago are no longer acceptable in journals that publish 10% of submissions.

    Another thing–when I review for top journals, I am given reviewer instructions that specifically charge me to look for reasons to reject. When I review for third tier journals, they encourage reviewers to help get the article from where it is to publication. The end product may ultimately be excellent, but instead of saying “buzz off, it’s not ready yet” the third-tier journal says “let’s see what we can do to make this great idea see the light of day.”

    Like

    Mikaila

    January 26, 2009 at 4:15 am

  2. “narrow audience, not due to quality”

    of course this raises the question that in some cases a narrow audience may reflect a failed paradigm and thus sometimes “narrow audience” is not exculpatory of the accusation of low quality, but synonymous with it. an interesting question would be to see if there are general principles for identifying high quality research agenda that (at least in the short to medium-term) nonetheless have narrow audiences. for instance, a substantive focus on a small population or intensely demanding and specialized methodologies might limit the appeal of intrinsically worthwhile research agendas.

    Like

    gabrielrossman

    January 26, 2009 at 4:29 pm

  3. Surely there are times when a dominant/senior reviewer or editor has favored a theoretical paradigm (consciously or not) or when theoretical paradigms that challenge largely accepted tenets of the discipline. Although peer review is different from the popular publishing process, there are plenty of stories of classic works of literature now regarded as masterpieces that were initially rejected by multiple publishers. There is a level of capriciousness to the process that almost definitely allows for top-quality work to slip through the cracks. It would be interesting to know how many theoretical works that were initially rejected from top-tier journals went on to become the dominant paradigm.

    Like

    Trey

    January 26, 2009 at 5:33 pm

  4. I think we are missing the whole problem of hegemonic vs subaltern paradigms/fields/areas of study. New areas of study, alternative modes of analysis, etc. don’t usually ‘make the cut’ when reviewed by main journals. It takes some time for the subalterns to become part of the hegemonies, and in that time those people still need to be published and read.

    I think we’ll more likely see a transformation of journal use and reading as the ability to get single electronic articles begins to transform people’s reading habits more than the arrival of the ‘disciplinary journal of choices’ selection of best. I find myself reading less by their editors version of ‘quality’ than by the version of quality defined by ‘cited or recommended by peers’.

    Like

    jeremy hunsinger

    January 26, 2009 at 6:03 pm

  5. The tier system of journals is essentially a reputation ranking system, and as well know there are significant lags in reputation rankings. And so even though some journals may be considered third-tier by the majority of the field, it’s possible that the quality of the work is now on par with other specialty journals normally afforded second tier status. I think this is probably now the case with Mobilization, the specialty journal for people who study social movements.

    Like

    brayden

    January 26, 2009 at 6:32 pm

  6. Mobilization certainly has moved up in esteem, and fairly rapidly for having been established in the mid-90s. I would argue that it is the top SM specialty journal now.

    But SM has a wide audience which may have allowed its growth, as Gabriel points out. For instance, the Journal of World Systems research is specialized, adopted by an ASA section, but has a relatively narrow (and shrinking) audience, and perhaps remains low ranked for that reason.

    Like

    cwalken

    January 26, 2009 at 9:46 pm

  7. BTW, Omar wrote a post a while ago about the assessment problem that small generalist journals face. His post is relevant to this question as he makes the case for why generalist journals that aren’t part of the top 2 or 3 get classified as third tier.

    Like

    brayden

    January 26, 2009 at 10:49 pm

  8. Probably best to adopt an agnostic stance towards all the work, irrespective of where you find it.

    Relying on the reputation of a journal to evaluate the work that appears therein is a heuristic at best. We ought to be given to more effortful processing than this.

    Like

    James

    January 26, 2009 at 11:37 pm

  9. Pity the sociologist from a small country, who, shock, might care about sociology in that country.

    Like

    Aileen O'Carroll

    January 26, 2009 at 11:48 pm

  10. Omar wrote a post?! Where is he, anyway? (Someone direct him back into the blogosphere from whatever Baudrillardian simulcrum he’s been in.)

    Like

    tf

    January 27, 2009 at 12:20 am

  11. One might establish a continuum between publishing in a top journal and writing a blog-post (or a comment to a blog post!). Third-tier journals and conference proceedings are probably near the middle of that continuum. But I don’t think there is any reason to hold a statement against the author simply on the basis of where it is published.

    For me, the important thing about top journals is their ability to represent mainstream consensus. That’s an important thing to establish in your writing, and it is much easier to do using top journals. So, for example, if you want to say things like “Organization theorists are increasingly drawing on social movement research to frame their research questions. They focus especially on…” it is useful to find references to SM in journals like Organization Science and ASQ, especially, of course, when there’s a nice special issue to cite.

    More peripheral journals/sites are less representative of major developments. Or, perhaps just, less easily representative.

    Like

    Thomas Basbøll

    January 27, 2009 at 5:02 pm

  12. I doubt that the category as Fabio’s defined it hangs together. Student-run journals in particular seem like a different case to me.

    The specialty journal of an unpopular specialty really can be very high-quality– the small population has an active commitment to a research agenda or methodology, and takes advantage of hothouse cultivation to push arguments forward and improve them rapidly. Not every hothouse plant can survive transplantation to the outside world, but some do. And the practitioners of the unpopular subfield may be able to make a lot more practice a lot more quickly if they don’t have to justify their whole approach to skeptical editors and referees with every article. If the research agenda turns out to have been a fruitful one, then the high-quality work in those journals will turn out to have been important ones.

    The low-tier generalist journals can provide a useful outlet for “here’s a true thing I discovered, that may not be revolutionary and doesn’t reshape any paradigms but which adds to the sum total of human knowledge.” Even top scholars often have such base-hit papers, and it’s a legitimate and useful research career (though not an elite research career) for someone to have a lot of them.

    The student-run journals, however, are basically an extension of the undergraduate extracurricular activity model into a domain where it doesn’t belong.

    Like

    Jacob T. Levy

    January 27, 2009 at 5:42 pm

  13. I think lower-tier journals can, though this obviously is rare, also publish “here’s a true thing I discovered, that may indeed be revolutionary, but does not fit into the existing paradigms”-types of papers. (Though, at the moment, no examples come to mind.)

    In the end, most areas of research have so many ‘strong’ outlets that good work ends up finding its way into one of the ‘top’ journals, even though peer review obviously can be noisy (particularly since the difference between an R&R and rejection, more often than not, is completely left to editorial judgment, rightly so—————most reviewers reject papers and thus counting ‘votes’ does not, thankfully, work as an editorial strategy).

    Like

    tf

    January 27, 2009 at 6:09 pm

  14. Isn’t “revolutionary but does not fit a paradigm” actually a pleonasm?

    Like

    Thomas Basbøll

    January 27, 2009 at 6:43 pm

  15. Being a blogspot on orgtheory, I was expecting to see some ecological and institutional arguments in responses to the questions posed by Omar. However, most arguments were about the institution of academia (which is loosely defined, or rather undefined), which sets standards for what is cool and what is not, but no one seem to acknowledge the ecological dynamics. Well, these “Third Tier”, as the academia dubs it, journals exist, simply because there is a niche for such journals, which remains rather vast, considering that over 50% of all articles submited to the 1st and 2nd tier journal gets rejected and many authors of such rejected articles might seek a home for their articles, even if such publications are not rewarded by the “ACADEMIA”. In addition, it is difficult for any journal to become a top tier journal from day one. Strategic Entrapreunership Journal of the strategic management society claims that it intends to be a ‘A’ journal like its sister journal the SMJ. But, how many “TOP TIER” scholars consider this journal as their journal of first preference? Or how many “TOP TIER” B-Schools recognize this journal among its A list. Similar is the case with another new journal in the intersection of Strategy and Organizations. It certainly intends to publish GOOD articles, it probably even keeps up to that intend. However, I happened to talk to the journal’s founding editor in a consortium and he openly says that he advices his students that any article that the student considers not worthy of an ASQ space should not be written on the first place. So, there is a stigma attached to these “Third Tier” journals, but there is also a large audience among the academic community, which desperately seeks out to these journals when their articles exhausts all or most “TOP TIER” outlets, just to get some closure to their work. Now, that is an answer to the question, “Why do third tier journals continue to exist”. But, it doesnt say, why they come into existence on the first place. As I had mentioned, no journal editor starts a journal saying, I am going to start a BAD journal. The founder’s hubris could explain part of the reason. It is possible that the founder of the journal considers himself as a reputable figure in the academia, but who doesnt see his worth recognized by the current editors of journals that probably have rejected a number of his recent articles or have not given him a chance to edit their journal. Hence, he or she seeks to create new “TOP TIER” journal as a means of enhancing his status. However, once operational, journal management might become too much of a responsibility that the founder loses interest in and the journal loses its intended rigor.

    Like

    Rajiv Krishnan Kozhikode

    January 27, 2009 at 7:08 pm

  16. On a different note, isnt Havard Law Review a student run journal?

    Like

    Rajiv Krishnan Kozhikode

    January 27, 2009 at 7:16 pm

  17. The student-run journals, however, are basically an extension of the undergraduate extracurricular activity model into a domain where it doesn’t belong.

    It’s good to finally see someone prepared to say what needs to be said about legal scholarship.

    Like

    Kieran

    January 27, 2009 at 7:21 pm

  18. Agreed!

    Like

    tf

    January 27, 2009 at 8:35 pm

  19. Yes, ha ha, and yes, it’s importantly true of the system of student-run law reviews as a system. But (just to be crystal-clear because you never know when something locked in google’s guts will come back to bite you on the hindquarters) I trusted that everyone recognized that when FRo included “student-run journals” as a subset of “third-tier,” he was likely excluding the universe of legal scholarship.

    Like

    Jacob T. Levy

    January 27, 2009 at 9:13 pm

  20. I am glad to see this topic brought up.

    @teppo: Most reviewers reject articles? So editors over turn, or they go with one R&R over two rejects for their own reasons?

    Like

    Jordi

    January 28, 2009 at 6:54 pm

  21. Well, at the top journals some 90% of pieces (perhaps 80) get rejected, right? But, then—-based on my discussions with an A editor—-apparently the difference between those that get an R&R versus those that barely get rejected is NOT in what the reviewers have, in aggregate, categorized the piece as (again: mostly reject) rather in the judgment call that the editor makes.

    Like

    tf

    January 28, 2009 at 8:12 pm

  22. Sure. Now I understand. It could also be that 90% get rejected but reviewers like all of them and it is space considerations that lead to rejection rate. What you are saying is that reviewers are exacting and editor has more to do with which articles make it in.

    Like

    Jordi

    January 28, 2009 at 9:34 pm

  23. […] Orgtheory asks what the hell the point of third tier journals is. I have an old publication lying around in a third-tier journal because they, well, paid me. […]

    Like


Comments are closed.