academic moneyball
Fabio
There’s been quite a bit of blogging about Dan Drezner’s recent post about Moneyball and academia. For those who don’t know, Moneyball is a book by Michael Lewis that shows how Oakland A’s manager Billy Beane used statistical analysis to identify over and under valued baseball players. For example, he discovered that getting on base leads to runs (and wins) more so than batting average. Basically, Billy Beane used evidence based management. The main lesson: recruit players using statistics that you can show are related to performance.
Drezner talks about how some academic programs have adopted a “moneyball” approach by hiring quality people in unusual or unpopular specialties. Drezner sees the value, but says that downsides are that you can be poached and that you be overspecialized. The first criticsm is odd, since poaching is a sign that you have valuable workers. The second criticism isn’t a problem either: a lot of programs would have a better profile as specialists rather than undistinguished generalists. Since I teach Moneyball in my class, I have given much thought to the academic moneyball strategy. Here are my thoughts:
- For programs with no PhD program, or ranked below 50 or so, Moneyball is probably your only chance at moving up. Why? If someone can clearly signal quality in popular specialties, they will probably be scooped by a higher ranked program early in the career. Going head to head with research 1 scools is simply a losing proposition. This strategy *can* be successful. A number non-research 1 universities have built highly regarded programs in unfashionable specialties: GMU & free market economics & Souther Illinois @ Carbondale philosophy is a center of Dewey scholarship.
- For large programs ranked > 50 or so, Moneyball can be tricky. If you correctly predict what is unpopular now, but popular in the future, you could hit the jackpot (see the Duke english dept circa 1990). That big batch of weirdos becomes the center of gravity for the whole profession in about 5-10 years. If you guess wrong, then you get … well, a big batch of weirdos!! And unlike baseball, you can’t trade unproductive scholars. I omit names to protect the guilty.
- For small, highly ranked programs, you are required to play Moneyball all the time. There is no point in trying to grab all the great players, just concentrate one one or two specialties. In sociology, programs like Johns Hopkins concentrate in world systems and soc of education. I don’t know anyone there, but my hunch is that it is too difficult to chase every hot young PhD and compete with other programs. Best to built on your specialty.
Interested folks can also read an old post on Moneyball, football and management over at Marginal Revolution.
orgtheory.net 
An enormous difference between Moneyball in baseball and here in academia is that in baseball the teams actually play one another. It’s in the face of playing each other, that great commensuration metric, that the effectiveness of strategies is revealed.
It’s not really clear what you get out of your strategy by building on specialties, except that it makes it easier to recruit in your speciality (which is a big difference). But building on specialties has huge costs if one thinks there is a relationship between the narrowness of a search and the quality of a hire.
In any case, the real Moneyball players in all this are highly ranked public universities.
LikeLike
Jeremy
October 13, 2006 at 10:13 pm
I think the moneyball strategy does a great job of describing some recent trends in academia. Other specialists programs in sociology that have enjoyed a measure of success by concentrating on somewhat obscure areas (i.e. experimental social psychology) are Iowa, South Carolina and NC Charlotte. In the late 70s and early 80s Princeton University was a rather run of the mill and under-performing program. They decided to specialize in the then obscure subjects of economic sociology and the sociology of culture (hiring people such as DiMaggio, Wuthnow and Zelizer). They guessed right and became an elite program by the early 1990s. That’s probably the most clear example of the moneyball strategy in recent history in sociology.
It seems that beyond pursuing a specialist strategy, if we take the moneyball simile seriously, a sociology program has to adopt a strategy of evaluating talent that is different from what is pursued by the majority. This would entail making risky tenure choices and taking chances on scholars that do not currently enjoy a large measure of reputational success in the mainstream of the discipline, or who do work that is not necessarily “elite” at the moment but which has the potential to become so if the mainstream adjusts and comes to meet them in the middle.
I think this requires a fairly good deal of vision on the part of the administration and a willingness to make a mistake and fall flat in your face. My guess is that given the inertia and conservative group-think that characterizes the way that most sociology programs are run, this requires a certain confluence of special circumstances. One recent success story that comes to mind in sociology is Jeremy’s PhD alma mater (I heard that a certain someone also works there), a program that has recently shot up the rankings primarily based on hiring young promising faculty that do work that is not necessarily mainstream, but that is interesting and edgy (it’s all about OPS and OBP! Forget about average and RBIs! Go Mets!).
LikeLike
Omar
October 13, 2006 at 10:31 pm
Some depts. may look like my Giants team. Old and average productivity. Brian Saebean has moved in the opposite direction of the Bluejays and A’s, trying to find old guys that no other team is willing to pay serious money for anymore. I suppose he thinks that he’s doing arbitrage of a different sort. The end result though is mediocrity. As a GM he survives solely on reputation (well, and by staying in contention, although rarely finishing at the front of the race).
LikeLike
brayden
October 13, 2006 at 11:32 pm
Malcolm Gladwell has a review of “The Wages of Wins”, which seems to recommend a Moneyball-type approach to figuring out a player’s worth.
Coming to this post, I’m not sure how the Moneyball analogy applies to academics. What is the equivalent of recruiting “players using statistics that you can show are related to performance”?
The examples you — and Drezner — have given seem to me to be *guesses* about which of he un-glamorous subfields are likely to become hot and sexy tomorrow. I don’t see in them any ‘Moneyball technique’ (a strong reliance on statistics, an evidence-based identification of parameters relevant to ‘winning’, etc).
LikeLike
Abi
October 14, 2006 at 2:37 pm
I dont think the techniques are the same but Fabio and Dan are suggesting that in academia organizations also look for arbitrage opportunities. That is one of the basic themes of Moneyball. I think the way that you determine the vaule of an asset is rather specific to the particular industry.
LikeLike
brayden
October 14, 2006 at 3:00 pm
A lot of good coments. A few quick responses:
1. Jeremy said: “An enormous difference between Moneyball in baseball and here in academia is that in baseball the teams actually play one another.”
I think you underestimate how much depts compete, though I do admit it is not quite the same as sports. Depts directly compete for grad students and faculty hires. They indirectly compete for space in journals, grants, reputation and coverage in the mainstream media.
Jeremy also said: “It’s not really clear what you get out of your strategy by building on specialties, except that it makes it easier to recruit in your speciality (which is a big difference). But building on specialties has huge costs if one thinks there is a relationship between the narrowness of a search and the quality of a hire.”
I agree – become too narrow and you risk a lot. You simply need a certain size to build speciaization. But, on the other hand, if you can take risk – as Brayden said – you can become known as a player in the field. It’s not a strategy for a generalist program like UCLA or my own employer (Indiana), but I could imagine other schools taking the risk and getting a big payoff.
2. Abi said: ” Coming to this post, I’m not sure how the Moneyball analogy applies to academics. What is the equivalent of recruiting “players using statistics that you can show are related to performance”?”
Here’s an example. Currently, most sociology departments evaluate professors on pubications in two journals – the American Sociological Review and American Journal of Sociology. The top 20 programs are always in a fighting for young PhD’s who have managed to publish in these two venues However, say you are program rank #52 and you can’t pay the salary needed to attract a young hot shot.
What do you do? Well, the Moneyball lesson is that you might figure out what makes a scholar well known aside from ASR or AJS publication. Then you try to scoop that person up before they make it big. Is it easy? No, and it entails risk. But if you guess right, you can boost you reputation by thinking outisde the box.
Abi also said: “The examples you — and Drezner — have given seem to me to be *guesses* about which of he un-glamorous subfields are likely to become hot and sexy tomorrow. I don’t see in them any ‘Moneyball technique’ (a strong reliance on statistics, an evidence-based identification of parameters relevant to ‘winning’, etc).”
They aren’t guesses, they’re success stories – GMU has probably become the leading heterodox econ program, when they started 20 years ago no one had heard of them; Illinois @ Carbondale is now the undisputed place to study Dewey and pragmatism, which is now a *huge* topic in philosophy and cultural studies; Hopkins has remained a top 20 program by concentrating in world systems and education; and Duke English, by poaching scholars not fully appreciated by the mainstream, jumped from rank 30-ish to top five in about ten years. Rochester went from no-name poli sci program to top 10 by exclusively doing rational choice.
Omar also pointed out that a few notable sociology programs did well to invest in experimental social psychology, which has become a well known area in sociology.
In all of theses cases, administrators had to take a very serious risk. They had to take a guess about undervalued fields. Doesn’t always work, but remember the alternative: compete with Harvard for the best mainstream scholars. If you do that, you’ll always come in second. The top research 1 programs are the evil Yankees – a huge wallet and they pay top dollar for the best talent.
3. Finally, an empirical comment. If I were an academic manager, I would try to figure out the following, if I were playing Moneyball:
– which young PhD’s are likely to hit top journals/ quality field journals, aside from those you have already done so as grad students?
– which grad students, if any, will underperform as faculty, mainly because they got their publications by working with advisors?
– which scholars are one hit wonders? They get one awesome publication, but then publish little afterwards.
– which types of scholars are well known (appt in top programs, reputation) without traditional publication records?
If I were the dean of a school hungry for reputation, I’d pay top dollar for answers to these questions.
LikeLike
Fabio Rojas
October 14, 2006 at 4:07 pm
Brayden, Fabio: Thanks for your responses.
As I see the discussion, three words stand out: arbitrage, risk and guess.
A second observation: all the success stories are about correctly predicting subfields that would later become important. On the other hand, the empirical comment (‘… if I were playing Moneyball) has ideas about individuals who are undervalued today, but are likely to become hot property later. I think it’s better to deal with these two separately. Let me stick to the former.
What I see in the success stories is a set of guesses — certainly very good ones! — about some unsexy subfield which went on to become hot a decade later. For every such success story, wouldn’t there also be quite a few failures that followed a bad guess?
For Moneyball to be applicable, we need some statistically-relevant measures that may mitigate risk inherent in an arbitrage opportunity, and guide a Dean to a good (or a less bad) guess. I would really like to see the evidence for the use of such measures by GMU Econ, Duke English, or any of the other success stories.
LikeLike
Abi
October 15, 2006 at 7:47 am