orgtheory.net

ranking scholarship

If you’re unconvinced that academic rankings matter to organizational outcomes, you’ve never been associated with a professional school. I used to joke with Teppo about the size of the poster that hung in the glass doors of the Marriott School building announcing BYU’s ranking in BusinessWeek’s top 10 undergraduate business programs. (I can only imagine what calamity of modesty hangs from the doors now that BusinessWeek ranked BYU 22nd among MBA programs!) Business, law, and medical schools care greatly about where they’re ranked because ranking is tightly connected to revenue streams. The relevance of rankings changes the schools’ administrative practices and policies; they adapt what they do to excel according to the criteria of the ranking systems. This behavioral transformation is one of the main points made by Michael Sauder and Wendy Espeland in their research on rankings and reactivity (see these past posts about their work).

Is there a similar kind of reactivity influencing the kind of scholarship academics do? Do we care so much about which journals are ranked highly that we choose research topics, theories, and methods that would help us get published in those journals? These are questions that came to mind when skimming this interesting exchange in the recent issue of the Academy of Management Learning and Education – the Academy journal that publishes essays about the practices of business education and scholarship. The lead article by Nancy Adler and Anne-Wil Harzing presumes that the current ranking system does not provide incentives to management scholars to do research about “questions that matter most to society.” The leading organizational and management journals (e.g., Administrative Science Quarterly, Academy of Management Journal, Organization Science), they argue, do not promote research that benefits the public good. And because business schools tend to reward scholars for publishing in these journals, our literature lacks the public impact it might otherwise have. They quote the former president of the Academy of Management, Steven Kerr, who said that we have mistakenly embraced the practice of “of rewarding A [publications in a narrow set of top-listed journals] while hoping for B [scholarship that addresses the questions that matter most to society].”  Adler and Harzing (somewhat ironically) use institutional theory to support the idea that the norms of the field pressure schools to become isomorphic in their estimation of journal quality and to reward scholars accordingly.

A number of scholars, including the dean of the Kellogg School, respond to Adler and Harzing and based on a quick skim, the responses are cautionary when engaging with A&H’s idea that we should put an immediate moratorium on all rankings. One of my first responses to A&H’s argument is that, unlike professional school rankings, scholarly output rankings don’t really exist in a formal sense. There may be lists of top journals (e.g., ISI’s Web of Knowledge) but journals are not formally ranked in the same way that schools are. I suspect that schools would actively resist such a ranking. There’s a benefit to having some ambiguity about what is a top journal. It allows local elites (i.e., those who are dominant in any one school) to assert their own definition of status (which itself revolves around the field’s definition of status). Thus, at school X, the status hierarchy of journals will have a more disciplinary feel, while at school Z the hierarchy is more firmly rooted in the management journals. Having a slippery, informal, and local status system allows departments to make tenure decisions based on its particular personality rather than on a strict count of publications (granted, some schools have formalized their review process to the point that counting publications is the most important way of assessing scholarly quality).  The fuzziness of the status hierarchy also gives more weight to outside letter writers, who can provide a quality assessment from the point of view of the scholar’s audience and peers.

There is a more general point of disagreement I have with the claim that current management scholarship matters little to society. This claim, which has been made by many others (see, for example, Teppo’s post about our “theory fetish”), typically privileges one world view over another and is asserted when one theory or one topic of interest is not getting as much attention as the claimant would like. I cringe when I hear management scholars argue that we need to become more relevant because what they often mean is that our scholarship should have a greater impact on the way business is done. While I can’t disagree with the idea that I’d like the business community to read and care about my research, there is a lot of research out there that I’d prefer the business community never heard of at all. Further, my idea of improving the public good is probably somewhat different than views held by other scholars. Rather than getting in a shouting match about which values our top journals should promote, I think the academic community (and the nonacademic community) is better off if we leave it to journals to figure out what is high quality scholarship and then let the market of ideas take over to sort the relevance of those ideas.

Written by brayden king

March 27, 2009 at 7:47 pm

6 Responses

Subscribe to comments with RSS.

  1. “let the market of ideas take over to sort the relevance of those ideas” … agree !

    I think there’s a strong parallel with the exploration-exploitation dilemma that organizations face: Should research be (imemdiately) relevant – and therefore exploit-able ? Or should it be more exploratory – with quality of scholarship being the value metric rather than a loosely defined metric such as “relevance”?
    (cant wait to start grad school!)

    Like

    Sd

    March 27, 2009 at 9:04 pm

  2. There are many good points here, but I’m a little confused. Are you saying that the determination of “quality scholarship” can and should be separated from the determination of “relevance” and/or “values”?

    And that journals should determine the former and individuals the latter?

    Like

    Michael Bishop

    March 27, 2009 at 9:09 pm

  3. Sd – good point. Having an open journal system that doesn’t prioritize specific values creates the conditions for idea competition. We don’t need to constrain our exploratory habits more than they already are.

    Michael – yes, you said it better than I could. Journals should determine scholarship quality and individuals/the public should determine which are relevant to practice. Trying to enforce a standard of practical relevance at the front end will constrain diversity and quality, IMHO.

    Like

    brayden

    March 27, 2009 at 9:13 pm

  4. Brayden, you have a real knack for asking hard questions…

    Let me add one comment: in schools of education, there is a clear distinction between “social studies of education” and the “practice of education.” Maybe B-schools, in presenting themselves, could benefit from the same jargon. Perhaps if the curriculum were more clearly labeled “social studies of administration” vs. practice, then you’d get less argument.

    Like

    fabiorojas

    March 28, 2009 at 8:12 pm

  5. Re: “let the market of ideas take over to sort the relevance of those ideas”

    Well… true, but this only happens if the market really operates under the classic free market assumptions – no informational asymmetries, so that everybody has access to all publicly available information (i.e. knowledge production in all journals). Moreover, market participants should have all the time in the world to read and decide what is significant, relevant etc. But markets are socially constructed. Social construction implies the meaning of what is relevant is socially constructed too. If status (of the journal, of a school) becomes a signal of higher quality, relevance whatever, then probably it is worthwhile worrying how this ranks are constructed and what is their implication for the disemmination of knowledge.

    Along this lines, an observation/ question: I live in one of the world capitals and it happens that I receive announcements for public conferences from 3 of the top schools of the world (speaking of ranks). Interesting enough, non-academic speakers aside, the talks which are over-subscribed are those offered by professors with publications in A-ranked journals. This does not imply causation – however, correlations of this sort sometimes trouble me.

    Like

    Dada

    April 2, 2009 at 9:38 am

  6. […] 2 comments For previous orgtheory posts on college rankings: b-schools, sociology PhD programs, b-schools again,Sauder on the […]

    Like


Comments are closed.

%d bloggers like this: