Archive for the ‘productivity and performance’ Category
“there’s no rankings problem that money can’t solve” – the tale of how northeastern gamed the college rankings
There’s a September 2014 Boston.com article on Northeastern University and how it broke the top-100 in the US News & World Report of colleges and universities. The summary goes something like this: Northeastern’s former president, Richard Freeland, inherited a school that was a poorly endowed commuter school. In the modern environment, that leads you to a death spiral. A low profile leads to low enrollments, which leads to low income, which leads to an even lower profile.
The solution? Crack the code to the US News college rankings. He hired statisticians to learn the correlations between inputs and rankings. He visited the US News office to see how they built their system and bug them about what he thought was unfair. Then, he “legally” (i.e., he didn’t cheat or lie) did things to boost the rank. For example, he moved Northeastern from commuter to residential school by building more dorms. He also admitted a different profile of student that wouldn’t the depress the mean SAT score and shifted student to programs that were not counted in the US News ranking (e.g., some students are admitted in Spring admissions and do not count in the US News score).
Comments: 1. In a way, this is admirable. If the audience for higher education buys into the rankings and you do what the rankings demand, aren’t you giving people what they want? 2. The quote in the title of the post is from Michael Bastedo, a higher ed guru at Michigan, who is pointing out that rankings essentially reflect money. If you buy fancier professors and better facilities, you get better students. The rank improves. 3. Still, this shows how hard it is to move. A nearly billion dollar drive moves you from a so-so rank of about 150 to a so-so rank of about 100-ish. Enough to be “above” the fold, but not enough to challenge the traditional leaders of higher ed.
So far, the Patriots have been nailed on two cheating scandals – deflation gate 2015 and the 2006 spying scandal. Each of these is interesting in its own right but there is one implication that few are willing to utter. The Patriots are probably cheating in more ways than we imagine.
The intuition is simple. Cheating incidents are not independent. It is not likely that every person will cheat with equal probability. Rather, people who want to cheat are the most likely to cheat and do so over and over. Also, consider incentives. They have been caught cheating multiple times and that hasn’t seemed to harm them much at all. The conclusion is that it is highly likely the Patriots are cheating in other ways.
I think it would be interesting for the fans of vanquished teams to conduct Levitt style analyses of the Patriots. I would guess that looking at other data in addition to the now famous fumble analysis will yeild some interesting answers.
Higher education has become dependent on human capital arguments to justify its existence. The new gainful employment rule for for-profit colleges, announced yesterday by the Obama administration, reminded me of this. It clarifies what standards for-profits have to meet in order to remain eligible for federal aid, which makes up 90% of many for-profits’ revenues.
Under the new standard, programs will fail if graduates’ debt-to-earnings ratio is over 30%, or if their debt-to-discretionary-earnings (income above 150% of the poverty line — about $17,000 for a single person) is over 12%.
Now, we could have a whole other conversation about this criterion, which is really, really, weak, since it no longer takes into account the percent of students who default on their loans within three years. By limiting the measure to graduates, it ignores, for example, the outcomes of the 86% of students who enroll in BA programs at the University of Phoenix but don’t finish in six years — most of whom are taking out as many federal loans as they can along the way.
But I want to make a different point here. More and more, we are focused on return on investment — income of graduates — as central to thinking about the value of college.
Last week, discussed how police interact with urban communities and the hypothesis that police are rewarded for focusing on the drug trade and they are less rewarded for just keeping the peace. Once commenter asked: what are the goals of the police? What are they trying to maximize?
Upon reflection, I realize that I had no idea, but I could generate some hypotheses:
- Department budgets
- Violent crime
- Non-violent crime
- Property crime
- Victimless crime
- Social control (i.e., controlling specific populations)
- “Broken windows” – making certain locations look desirable
- Votes (for D.A.’s especially)
I’d be interested in any data show the relative importance of these goals, say, in police budgets, arrests, prosecutions, police hours, etc. Criminology readers – how would you rank these goals given your knowledge of the field?
Do people know about social impact bonds? I hadn’t heard of them till recently. Since then, though, I’ve developed a train-wreck fascination. They have the potential to combine all the worst features of the public and private sectors. And they can be securitized, to boot!
Let’s take a step back. What is a social impact bond, anyway?
Well. Imagine you have a social problem you’d like to solve. Say that you want to reduce recidivism among young people in prison. That sounds good, right? The problem, of course, is that taxpayers don’t want to pay for rehabilitative programs, and there’s lots of disagreement about what kind of program would actually help solve the problem, anyway.
The government says, Wouldn’t it be nice if somebody would take care of this for us, and we’d only have to pay them if they actually succeeded?
Enter Goldman Sachs.
A recent article in the Journal of Economic Perspectives reports a recent attempt to curb grade inflation. High GPA departments at Wellesley College were required to cap high grades. The abstract:
Average grades in colleges and universities have risen markedly since the 1960s. Critics express concern that grade inflation erodes incentives for students to learn; gives students, employers, and graduate schools poor information on absolute and relative abilities; and reflects the quid pro quo of grades for better student evaluations of professors. This paper evaluates an anti-grade-inflation policy that capped most course averages at a B+. The cap was biding for high-grading departments (in the humanities and social sciences) and was not binding for low-grading departments (in economics and sciences), facilitating a difference-in-differences analysis. Professors complied with the policy by reducing compression at the top of the grade distribution. It had little effect on receipt of top honors, but affected receipt of magna cum laude. In departments affected by the cap, the policy expanded racial gaps in grades, reduced enrollments and majors, and lowered student ratings of professors.
My sense is that this shows that grade inflation, whatever its historical origins, acts as a competitive advantage for programs that few other market advantages. If you don’t have a strong external job market or external funding, then you can boost enrollments via grade inflation. It also absolves programs by masking racial under performance. The lesson for academic management is this: If you have inequality in funding, departments will compensate by weak grading. If you have inequality by race, departments will compensate by weak grading. Thus, academic leaders who care about either of these issues should implement policies where departments don’t choose standards and are accountable for results.
Over at Scatterplot, Jeremy’s been writing about his life gamification experiment, which involves giving himself points for various activities he’d like to be doing more of. I find this sort of thing totally compelling and have to admit I’m now giving myself all sorts of points in my head. (Finish unpacking one box — 5 points! Send an email I’ve been procrastinating on — 5 points!) Although not in 100 million years could I get my husband to play along with me, even for brunch, of which he is fond.
Anyway, the game brought to mind this post from Stephen Wolfram, in which Wolfram presents a bunch of data from the last 25 years of his life. Here, for example, are all the emails he’s sent since 1989. (Note the sharp time shift in 2002, when he stopped being completely nocturnal.) He’s also got keystroke data, times of calendar events, time on the phone, and physical activity.
Fascinating to read about, but perhaps not terribly healthy to pursue in practice. Although in Wolfram’s case, it sounds like he was mostly just collecting the data, not using it to guide his day-to-day decisions. Others become more obsessive. I don’t know if David Sedaris has really been spending nine hours a day walking the English countryside, a slave to his Fitbit, or if he’s taking poetic license, but it’s a heck of an image.
Clearly there are a lot of people into this sort of thing. In fact, there is a whole Quantified Self movement, complete with conferences and meet-up groups. One obvious take on this is that we’re all becoming perfect neoliberal subjects, rational, entrepreneurial and self-disciplined.
For me, though, what is fun and appealing as a choice — and I do think it’s a choice — becomes repellent and dehumanizing when someone pushes it on me. So while I’ll happily track my work hours and tally my steps just because I like to — and yes, I realize that’s kind of weird — I hate the idea of judging tenure cases based on points for various kinds of publications, and am uneasy with UPS’s use of data to ding drivers who back up too frequently.
It’s possible that I’m being inconsistent here. But really, I think it’s authority I have the problem with, not quantification.