orgtheory.net

Sociology Department Rankings for 2013

Update: I updated these analyses (fixing the double-counting problem). The results changed a little, so reload to see the new figures.

Last week we launched the OrgTheory/AAI 2013 Sociology Department Ranking Survey, taking advantage of Matt Salganik’s excellent All Our Ideas service to generate sociology rankings based on respondents making multiple pairwise comparisons between department. That is, questions of the form “In your judgment, which of the following is the better Sociology department?” followed by a choice between two departments. Amongst other advantages, this method tends to get you a lot of data quickly. People find it easier to make a pairwise choice between two alternatives than to assign a rating score or produce a complete ranking amongst many alternatives. They also get addicted to the process and keep making choices. In our survey, over 600 respondents made just over 46,000 pairwise comparisons. In the original version of this post I used the Session IDs supplied in the data, forgetting that the data file also provides non-identifying (hashed) IP addresses. I re-ran the analysis using voter-aggregated rather than session-aggregated data, so now there is no double-counting. The results are a little cleaner. Although the All Our Ideas site gives you the results itself, I was interested in getting some other information out of the data, particularly confidence intervals for departments. Here is a figure showing the rankings for the Top 50 departments, based on ability scores derived from a direct-comparison Bradley-Terry model.

"Top 50."

The model doesn’t take account of any rater effects, but given the general state of the U.S. News ranking methodology I am not really bothered. As you can see, the gradation looks pretty smooth. The first real “hinge” in the rankings (in the sense of a pretty clean separation between a department and the one above it) comes between Toronto and Emory. You could make a case, if you squint a bit, that UT Austin and Duke are at a similar hinge-point with respect to the departments ranked above and below them. Indiana’s high ranking is due solely to Fabio mobilizing a team of role-playing enthusiasts to relentlessly vote in the survey. (This is speculation on my part.)

You can do other things with this data, too. Here are the results of a cluster analysis of the votes, which brings out some interesting similarities.

"Cluster analysis."

Finally, Baptiste Coulmont carried out his own pairwise-comparison survey of French sociology departments—at least until he was overwhelmed by the pressure exerted by people who thought the very idea of such a ranking was morally offensive—and presented a nice analysis of it on his blog. Inspired by that, here’s are the results of a principal components analysis of the voting patterns.

"Factor analysis."

The x-axis is more or less the overall ranking itself. The y-axis is harder to interpret. If I have time I can look a little more closely at that. The colors in the plot show how long people typically to vote for that department in comparisons, with Slow and Fast categories representing roughly the bottom and top quintles of the distribution of times. High-ranking but “Slow” departments are interesting here, as it suggests they might be difficult to place with respect to their peers.

About these ads

Written by Kieran

March 25, 2013 at 3:58 pm

11 Responses

Subscribe to comments with RSS.

  1. [...] at OrgTheory, Kieran Healy has released the results of the All Our Ideas survey of “best” sociology department. Head over there for methodological details and results. Kieran also presents some nice graphs of [...]

    Like

  2. At first, I thought Slow and Fast was the average time it takes to complete the PhD. I wonder whether it would not be more relevant?

    Like

    Leo

    March 25, 2013 at 4:30 pm

  3. Indiana broke the top 10!!!!!!! Woooo!!!!!!

    Like

    fabiorojas

    March 25, 2013 at 4:33 pm

  4. They also get addicted to the process and keep making choices.

    I wouldn’t be surprised if doctors saw a rise in carpal tunnel syndrome this month.

    Like

    tina

    March 25, 2013 at 7:43 pm

  5. Hey Org Theory posters. Congratulations to the schools that made the Top 50! I am a loyal reader of your blog, and a PhD candidate at a school that did not rank. I would be interested in a companion post or two about strategies for people graduating with PhD’s from un-ranked schools for getting hired in tenure-track positions. Just wanted to remind you that we’re out here, reading your blog, and working hard on our research and degrees, just like people from the ranked programs.

    I also plan to consult my copy of Grad Skool Rulz, since that’s something I can do on my own. For reference, I have ~6 points from the “Should I go on the job market” questions on pages 51 and 52 of the review copy. Thank you Fabio for the useful resource.

    Like

    EmilyKennedy

    March 25, 2013 at 7:43 pm

  6. Looks like a big winner was Notre Dame, which is up about 18 points from where you would expect based on US News rankings. Florida and Florida State are both the big losers, each down about 12 points from US News.

    Fun regression: take your US News ranking. Subtract one. Then multiple by 25. Compare that to your rank on the results page from the OrgTheory/AAI 2013 Sociology Department Ranking Survey. r2=.91.

    Like

    neal

    March 25, 2013 at 8:19 pm

  7. @EmilyKennedy you and I are in the same position and Fabio has some useful posts on this topic if you search the blog. Basically the great equalizers are things like publications and, perhaps to a lesser degree, grants.
    I won’t be on the job market for another year or two but I follow who is getting hired were pretty closely via the soc job wiki. From what I can tell some unranked PhD programs really emphasize prestige. In other words, they will hire a Michigan PhD with no publications over a Southwest State Tech (fake name for an unranked school) PhD with a strong publication record. Its simply the way that it is; some places probably won’t take you or take you seriously no matter what. Even if I were to publish a first-author paper in ASR I’m not sure that Ivy League U would take me seriously as a job candidate. I’ve come to this conclusion by looking at 1) who is being hired 2) were they are hired and 3) looking up their CVs.
    However, there are other unranked PhD programs that will hire candidates from unranked PhD programs; they are less sensitive to prestige and probably weight publications, demonstrated teaching ability, and overall fit more than program prestige.
    My evidence is pretty anecdotal and haphazard but I couldn’t find any research on the hiring effects of program prestige for newly minted PhDs in sociology. I’m sure the effect of prestige is strong but one way to differentiate yourself is through publications and grant funding. At least that I’m doing…I hope it works!
    BTW, I am truly thankful for Fabio’s blog and book. For me it’s been a window into elite practices and how academic sociology is institutionally structured that I could not have gotten anywere else.

    Like

    Silly Wabbit

    March 26, 2013 at 4:16 pm

  8. [...] Sociology Department Rankings for 2013 (orgtheory.wordpress.com) [...]

    Like

  9. To which “Boston” does this study refer, College or University?

    Like

    Michael Malec

    March 28, 2013 at 1:39 am

  10. Michael—whoops, looks like the code I wrote was a bit too aggressive cleaning the department labels. It’s Boston University. In the full table Boston College is a few places further down, in the 51-60 range.

    Like

    Kieran Healy

    March 28, 2013 at 1:51 am


Comments are closed.

Follow

Get every new post delivered to your Inbox.

Join 1,168 other followers

%d bloggers like this: