orgtheory.net

upsetting the rankings

Brayden

Ranking stuff helps us understand how something fits into its category (e.g. hospitals, movies). Rankings of organizations often translate into reputational capital – a kind of intangible asset that helps organizations get more customers, more satisfied employees, etc. The ranking ends up becoming a key component of a company’s reputation. Given the value that reputation has for organizations, you can see why organizations care a great deal about ranking systems. If you’re ranked highly, then your organization will probably support the current ranking system. If you’re struggling, you probably question the legitimacy of the rankings.

Dissatisfaction with rankings is causing a number of U.S. college presidents to call for a boycott of the U.S. News & World Report’s ranking of top colleges and universities. As this Washington Post article explains, some administrators are asking their peers to stop providing the magazine with the numbers used to produce the rankings.

“But why should we help U.S. News sell magazines?” asked Robert J. Massa, vice president for enrollment and college relations at Dickinson College…

“We believe these rankings are misleading and do not serve well the interests of prospective students in finding a college or university that is well suited to their education beyond high school,” read a letter sent from 12 school president…

This effort to disrupt the ranking process ties into a larger debate about evaluating the quality of college education. The U.S. News ranking relies mostly on data about endowment size and the SAT scores of incoming students. Not surprisingly, these data tend to reproduce the reputational positions of elite schools. Up-and-comers have a harder time gaining ground in a system like this. Matthew Yglesias points to a couple of interesting statements on the topic.

But the more general problem that should be of interest to organizational scholars is, how do organizations deal with ranking systems that do not favor the organization? Not enough has been written about this. Most reputation scholars are interested in creating better rankings, decomposing existing rankings, etc. The more interesting topic to me though is how organizations interact with the rankings as a socially constructed source of reputation. Rather than thinking about rankings as measuring some latent quality of an organization, we can/should think about rankings as a social construct that organizations have a part in creating and amplifying. Reputation doesn’t exist outside the organization. It actually has an effect on the organization’s own identity – how it defines itself relative to its peers. Threats to an organization’s reputation, then, can also be threats to an organization’s identity.

This is the point of an excellent ASQ paper written by Kimberly Elsbach and Roderick Kramer, “Members’ responses to organizational identity threats: Encountering and countering the Business Week rankings.” Elsbach and Kramer look at how eight top business schools responded to new rankings of their schools. The Business Week rankings were meant to weed out the pretenders from the elite of business schools by providing a single set of criteria that students and recruiters could use to evaluate a school’s success. While this looked good on the surface, many business schools had in the past defined themselves in very unique ways. The new ranking system forced them to re-define their identity according to criteria that valued national prestige among students and recruiters. The new “objective” evaluative criteria of the Business Week rankings were a clear threat to identity.

Elsbach and Kramer refer to this distress when dealing with imposed evaluative criteria as “identity dissonance,” or the “cognitive dissonance related to the disparity or inconsistency between members’ perceptions of their organization’s identity (e.g. their perception that their school is ‘a dynamic, still growing program’) and the identity attributed to it by the Business Week survey (the assertion that the school ‘is standing still’)” (452-54). The rankings posed two kinds of threats: 1) threats to core identity attributes and 2) threats to the school’s positional status (i.e. their reputation). Interestingly, members of schools that received lower rankings typically responded with disbelief or denial. Rather than submit to the new rankings, members of lower-ranked schools tended to emphasize identity characteristics that they saw as existing outside the ranking criteria. Members tended to affirm existing “cherished dimensions” of their identity that had been overlooked by survey respondents (458). If they were unhappy with their position in the rankings, members strived to find a new basis for comparison by recategorizing the organization. For example, if a school did not fare well in comparison to other elite, national schools, then they might recategorize themselves as a regional school and seek to establish a reputation in this new category. Or they might seek to categorize the school as primarily-research oriented, casting it in the same category as Stanford and Chicago.

One of the more interesting implications of Elsbach and Kramer, I think, is that ranking systems reconstitute an organizational field by forcing organizations within that field to define their identities relative to a particular set of criteria. Identities are reconstructed in opposition to (or in congruence with) those criteria. Organizations on the outside looking in may seek to challenge the criteria (as you see the administrators doing now), but they are probably more likely to have success if they simply recategorize their organization and find an alternative set of criteria that allow them to distinguish themselves from their peers.

Written by brayden king

May 22, 2007 at 4:27 pm

9 Responses

Subscribe to comments with RSS.

  1. There’s a lot to be said about this, Brayden. If you notice the US News college rating led to a successful attempt to create multiple categories – for example, public and private research schools now receive separate ranks, as do liberal arts.

    Also, a few schools have opted out of the system complete by refusing to participate. These tend to be funky liberal arts schools who have cornered a particular niche and can exist without the free publicity and/or legitimacy conferred by the rankings.

    But overall, I would guess that rankings probably force most orgs to focus on core identity, as the Elsbach research suggests, or to identify multiple population segments that require different rankings. The average person/org isn’t innovative enough to create a sustainable identity outside the ranking criteria.

    Like

    Fabio Rojas

    May 22, 2007 at 6:48 pm

  2. Man I need to revise my philosophy rankings paper and get more data.

    Like

    Kieran

    May 22, 2007 at 6:57 pm

  3. I’d argue that rankings can provoke one of three effects. As Fabio describes, they might champion a ‘competing logic’ and buck the rankings altogether (the ‘funky liberal arts colleges’), or they might regroup around a core identity (and describe themselves in terms of a niche.)

    These two responses, though, assume that universities’ identities are relatively fixed. Take a look at Gioia & Thomas (1996), and a third option is described: Universities can undergo organizational AND identity change, mimetically adopting practices prototypically associated with ‘top’ schools.

    I’d call this last impulse the most dangerous — the rankings end up creating (or at least conveying) an institutional norm that universities feel they have to subscribe to in order to appear legitimate, when in fact doing so may involve the abandonment of technically efficient practices or worthwhile values in favour of mimicry.

    Here’s the cit. btw –
    Gioia, D.A. & Thomas, J.B. (1996). “Identity, Image and Issue
    Interpretation: Sensemaking During Strategic Change in
    Academia,” Administrative Science Quarterly 41(3), 370-403.

    Like

    optimuscrime

    May 22, 2007 at 7:52 pm

  4. Many Canadian universities recently employed such a boycott of the Maclean’s rankings up here. It’s interesting how they got to a point at which the damage to their reputations of looking petty and defensive in their criticism of the rankings is less than the damage to reputations of being ranked low (or, in the case of a certain university, dropping suddenly in the rankings after several years of budget cuts pushed class size through the roof).

    Like

    Tina

    May 22, 2007 at 8:37 pm

  5. One sometimes unintended (but usually intended) consequence of rankings is to create order and commesurability where there previously was none (or more precisely where it was previously unacknowledged). Since fields are defined by contests as to what the dominant ranking principle in the field should be anyway, rankings may actually facilitate the struggle of dominated organizations to define an “alternative” form of capital as a counter ranking principle. All fields have a ranking of players and positions anyway, the difference is whether this ranking is left in its practical (implicit) state or is exposed and objectified for everyone to see.

    Lots of the hyper-reaction against ranking comes from players who benefit from the ranking principle being left implicit and who suffer when the raw hierarchy of the field is exposed to the public and objectified in a crass manner (this is most likely to happen in fields such as art or academia which define themselves as different from fields in which hierarchies are taken for granted such as for-profit fields).

    My sense is that as academia becomes more heteronomous and accustomed to an institutional logic closer to the market, the dominant players will become more comfortable with public rankings while dominated players will become more skillful at defining “ineffable” attributes that cannot be captured by such rankings and which of course they will claim to possess.

    Like

    Omar

    May 22, 2007 at 9:08 pm

  6. The president of Reed College wrote a great article in Atlantic Monthly a few years ago. I went to Grinnell (in Iowa). It’s very similar to Reed and am very sympathetic to Reed’s approach. I’d be happy if Grinnell went the same way. See: http://www.theatlantic.com/doc/200511/shunning-college-rankings

    I’ve seen some business school deans make some pretty screwy decisions in their quest to move up the rankings. The rankings have a big influence on applications to some (many?) schools. Such schools that also have a weak sense of mission have trouble “ignoring” the rankings. It seems pretty easy for an administration to lose sight of their committment to education and research to chase after some variables weighted heavily in the rankings.

    Like

    David G. Hoopes

    May 23, 2007 at 3:54 pm

  7. […] discussing the logic and effects of what Bruno Frey calls “evaluitis”, to the possibility of upsetting the rankings, and to strong and ever increasing social forces resulting from the logic of folksonomy. My guess […]

    Like

  8. […] Reactivity: How Public Measures Recreate Social Worlds” deals with a topic that was recently brought up in this forum (which tells you that we are teetering so close to the cutting edge that we are going […]

    Like

  9. […] try to tie their findings to the research on corporate reputation, I think there are some obvious implications.  Corporate reputation scholars have spent a lot of time assessing reputation as a kind of […]

    Like


Comments are closed.

%d bloggers like this: