“you can’t fire your way to finland”

Last week a judge struck down tenure for California teachers on civil rights grounds. (NYT story here, court decision here.) Judge Rolf Treu based his argument on two claims. First, effective teachers are critical to student success. Second, it is poor and minority students who are most likely to get ineffective teachers who are still around because they have tenure — but moved from school to school in what Treu calls, colorfully, the “dance of the lemons.”*

To be honest, I have mixed feelings about teacher tenure. I’d rather see teachers follow a professional model of the sort Jal Mehta advocates than a traditional union model. This has personal roots as much as anything: I’m the offspring of two teachers who were not exactly in love with their union. But at the same time, the attack on teacher tenure just further chips away at the idea that organizations have any obligation to their workers, or that employees deserve any level of security.

But I digress. The point I want to make is about evidence, and how it is used in policy making — here, in a court decision.

The judge based his decision on some specific pieces of evidence. First, he argues that good teachers matter to classroom outcomes. Treu cites Harvard economist Raj Chetty:

A single year in a classroom with a grossly ineffective teacher costs students $1.4 million in lifetime earnings per classroom.

And Thomas Kane, also a Harvard economist (but in the ed school):

Students in the [Los Angeles Unified School District] who are taught by a teacher in the bottom 5% of competence lose 9.54 months of learning in a single year compared to students with average teachers.

So far, so good. There is room for critique here — surely not everything Chetty said in his deposition made it into the decision, but as stated, this is misleading. The big Chetty et al. 2011 study on teacher quality and adult salaries reported that replacing a bottom 5% teacher with an average one would increase the “present value of students’ lifetime income by $267,000 per classroom.” $1.4m is the undiscounted figure. Moreover, Chetty et al. immediately note that that’s if you drop someone truly in the bottom 5%. But because annual data are noisy, dropping the bottom 5% based on a year of information would increase the present value of the class’s lifetime income by only $135,000 — or $5000 over each child’s life.

(I’m not even going into the fact that the lifetime earnings estimates are based on earnings as of age 28. Actually, the whole value-added teaching thing deserves its own post — I started writing this last night and didn’t finish because I got totally sucked into reading all the fights about value–added models.)

But, whatever. Sure, better teachers are better for kids. Not going to fight you there.

Next, though, the judge cites David Berliner, an educational psychologist at Arizona State, who testified for the state of California. He testified that “1-3% of teachers in California are grossly ineffective.”

Now, one might debate the teacher impact studies. But the number on “grossly ineffective” teachers, it turns out, was totally made up. Okay, maybe it was more of an educated guess. But as Berliner himself later explained,

I pulled that out of the air….There’s no data on that. That’s just a ballpark estimate, based on my visiting lots and lots of classrooms.

Yet it’s presented as real. In fact, the judge goes on to multiply it out:

Given that the evidence showed roughly 275,000 active teachers in this state, the extrapolated number of grossly ineffective teachers ranges from 2,750 to 8,250.

This is bad enough. But it’s the final piece of evidence — or rather, the lack thereof — that takes the cake.

While the claim that good teachers have an effect on kids requires high-end econometrics and a dataset of 2.5m students to support it, the final argument — that removing teacher tenure will result in the least-effective 5% of teachers being fired, while having no significant unintended consequences — apparently requires no evidence whatsoever.

I can imagine many, many scenarios in which the elimination of tenure does not result in the worst 5% of teachers being identified and let go. I can also imagine many, many effects such a policy might have which would outweigh even the perfect identification and removal of the bottom 5% of teachers. The most obvious, of course, is that the further deprofessionalization of teaching — making teachers’ jobs totally dependent on standardized test results — will, if anything, drive even more of the best and the brightest away from teaching.

As Linda Darling-Hammond says, “You can’t fire your way to Finland.”

* I was curious about the origins of this phrase. I wondered if it might have been cooked up by advocates, which is where most of the recent uses come from—“Waiting for Superman” seems to have given it a burst of popularity. But no, Larry Cuban was using it all the way back in 1984.