orgtheory.net

“you can’t fire your way to finland”

Last week a judge struck down tenure for California teachers on civil rights grounds. (NYT story here, court decision here.) Judge Rolf Treu based his argument on two claims. First, effective teachers are critical to student success. Second, it is poor and minority students who are most likely to get ineffective teachers who are still around because they have tenure — but moved from school to school in what Treu calls, colorfully, the “dance of the lemons.”*

To be honest, I have mixed feelings about teacher tenure. I’d rather see teachers follow a professional model of the sort Jal Mehta advocates than a traditional union model. This has personal roots as much as anything: I’m the offspring of two teachers who were not exactly in love with their union. But at the same time, the attack on teacher tenure just further chips away at the idea that organizations have any obligation to their workers, or that employees deserve any level of security.

But I digress. The point I want to make is about evidence, and how it is used in policy making — here, in a court decision.

The judge based his decision on some specific pieces of evidence. First, he argues that good teachers matter to classroom outcomes. Treu cites Harvard economist Raj Chetty:

A single year in a classroom with a grossly ineffective teacher costs students $1.4 million in lifetime earnings per classroom.

And Thomas Kane, also a Harvard economist (but in the ed school):

Students in the [Los Angeles Unified School District] who are taught by a teacher in the bottom 5% of competence lose 9.54 months of learning in a single year compared to students with average teachers.

So far, so good. There is room for critique here — surely not everything Chetty said in his deposition made it into the decision, but as stated, this is misleading. The big Chetty et al. 2011 study on teacher quality and adult salaries reported that replacing a bottom 5% teacher with an average one would increase the “present value of students’ lifetime income by $267,000 per classroom.” $1.4m is the undiscounted figure. Moreover, Chetty et al. immediately note that that’s if you drop someone truly in the bottom 5%. But because annual data are noisy, dropping the bottom 5% based on a year of information would increase the present value of the class’s lifetime income by only $135,000 — or $5000 over each child’s life.

(I’m not even going into the fact that the lifetime earnings estimates are based on earnings as of age 28. Actually, the whole value-added teaching thing deserves its own post — I started writing this last night and didn’t finish because I got totally sucked into reading all the fights about valueadded models.)

But, whatever. Sure, better teachers are better for kids. Not going to fight you there.

Next, though, the judge cites David Berliner, an educational psychologist at Arizona State, who testified for the state of California. He testified that “1-3% of teachers in California are grossly ineffective.”

Now, one might debate the teacher impact studies. But the number on “grossly ineffective” teachers, it turns out, was totally made up. Okay, maybe it was more of an educated guess. But as Berliner himself later explained,

I pulled that out of the air….There’s no data on that. That’s just a ballpark estimate, based on my visiting lots and lots of classrooms.

Yet it’s presented as real. In fact, the judge goes on to multiply it out:

Given that the evidence showed roughly 275,000 active teachers in this state, the extrapolated number of grossly ineffective teachers ranges from 2,750 to 8,250.

This is bad enough. But it’s the final piece of evidence — or rather, the lack thereof — that takes the cake.

While the claim that good teachers have an effect on kids requires high-end econometrics and a dataset of 2.5m students to support it, the final argument — that removing teacher tenure will result in the least-effective 5% of teachers being fired, while having no significant unintended consequences — apparently requires no evidence whatsoever.

I can imagine many, many scenarios in which the elimination of tenure does not result in the worst 5% of teachers being identified and let go. I can also imagine many, many effects such a policy might have which would outweigh even the perfect identification and removal of the bottom 5% of teachers. The most obvious, of course, is that the further deprofessionalization of teaching — making teachers’ jobs totally dependent on standardized test results — will, if anything, drive even more of the best and the brightest away from teaching.

As Linda Darling-Hammond says, “You can’t fire your way to Finland.”

* I was curious about the origins of this phrase. I wondered if it might have been cooked up by advocates, which is where most of the recent uses come from—“Waiting for Superman” seems to have given it a burst of popularity. But no, Larry Cuban was using it all the way back in 1984.

Written by epopp

June 18, 2014 at 3:00 pm

8 Responses

Subscribe to comments with RSS.

  1. Related, I also don’t get the jump between eliminating tenure and ending the (delightfully named) “dance of the lemons.” Ok, maybe we get rid of the bottom 5%. But wouldn’t we still expect the next worst 5% to end up teaching the poorest, most troubled 5% of kids? How is this supposed to change the organizational dynamics that lead to ineffective teachers being placed with the worst-off kids?

    Like

    Dan Hirschman

    June 18, 2014 at 3:13 pm

  2. @Dan – Once you truncate a distribution, you shift the mean up. The new bottom teachers will be slightly better, which might result is a modest improvement for the worst students.

    Like

    fabiorojas

    June 18, 2014 at 3:42 pm

  3. Maybe. But even if we ignore the problems of noisy data and the sort of mythic potential to produce change that this whole debate assumes that individual teachers have, there’s an issue of education being (to a greater or lesser extent) a positional good. If the worst-off students always get the worst teachers, and there is any kind of large variation in teacher quality, it’s still going to end up being an inequality-promoting system.

    Like

    Dan Hirschman

    June 18, 2014 at 3:45 pm

  4. And the presumption of the Chetty et al. study is that you replace the bottom 5% teacher with a median teacher. If you’re talking about disadvantaged schools, there’s little reason to think the replacement will be a median teacher. So even if you buy the logic of how these studies calculate impact, the actual effect would still be much smaller.

    Like

    epopp

    June 18, 2014 at 3:49 pm

  5. Dan, I think you should drop the focus on inequality and focus on absolute improvements. If we bump up reading by some non-trivial level by truncating the worst teachers, let’s do it. Inequality will remain, but absolute shifts matter. Would you turn down a $100 bill just because Mark Zuckerberg was still richer than you?

    It’s a slight exaggeration to say that the literature on teaching shows that the main way that schools help people is by having good teachers. Once you control for who teaches, school effects go away. The implication is that we need to shift the mean teacher.

    Like

    fabiorojas

    June 18, 2014 at 4:02 pm

  6. I think the judge’s decision, and some of the comments here, implicitly assume that by eliminating tenure the worst teachers will exit the profession thereby “truncating the distribution”. I am far from an expert on the labor market for educators, but I I’m not sure that this is a viable assumption.

    Indeed, it seems plausible to me that this might shift more “bad educators” to the worst performing school districts as they are forced out of higher performing school districts after their tenure is eliminated and they get the axe. Lower performing school will likely offer lower pay, less benefits, etc. and be more willing to fill positions with potentially ineffective educators.

    So I could imagine a multi-step process in which 1) poor teachers are forced out of “good schools” 2) they elect to stay in the profession and eventually 3) they end up at already low performing schools and 4) mean teacher quality stays the same but the social distribution in teacher quality is even more skewed that before.

    My scenario might not be plausible. For example, some teachers might exit the profession or not be able to secure new employment if they are forced out of tenured jobs. Indeed, some of them may be older teachers who will find new employment opportunities scare. Still, I don’t think its plausible to assume that axing tenure will cause poor teachers to leave the profession en masse. Maybe there is some research on career trajectories for teachers that could be relevant?

    Like

    Silly Wabbit

    June 18, 2014 at 5:02 pm

  7. @SW, I completely agree, although I don’t know about the research (surely it’s out there). The other big factor they seem to be overlooking is whether getting rid of tenure would change who is attracted to the profession.

    Like

    epopp

    June 19, 2014 at 12:38 pm

  8. @epopp- Yes I think thats plausible too. Anecdotally, I have a few very talented friends who could have been quite successful in other professions but chose to teach with job security, in addition to the intrinsic benefits of working with kids, as a major draw. Again, I’m not sure but I could imagine that eliminating tenure and eroding working conditions for teachers could truncate the right-hand side of the distribution and reduce the number of highly talented people who select into the profession.

    Like

    Silly Wabbit

    June 19, 2014 at 2:35 pm


Comments are closed.

%d bloggers like this: