“i’m awfully glad i’m a beta”: the educational effects of status groups

Thad Domina, Andrew Penner and Emily Penner have a really interesting new paper out in Sociological Science, AKA the official journal of people on my Twitter feed.

So it turns out that some crazy Californians had the great idea to assign high school students color-coded IDs based on their standardized test results. Everyone got a white, gold, or platinum card based on their level of proficiency on the California state tests. The students had to display their ID card whenever they were on campus, and gold and platinum card holders got certain perks, including discounts on school events, but most visibly an “express” line in the cafeteria. What could possibly go wrong?

Well, this raised some eyebrows once word traveled beyond the schools themselves. The article doesn’t name the two schools, so I won’t either, but Google exists, and it sounds pretty clear that creating these visible new categories had real status effects. You can find quotes about how kids in honors classes with platinum cards told other kids in honors classes who only had a gold card that they shouldn’t even be there, and a principal apparently told some girls they should try to find platinum prom dates.

The program was in place for two years, then was shut down after quite a bit of unfavorable press. But not before our intrepid sociologists managed to collect some data.

Domina et al. isn’t really about these kinds of categories at all but this is pretty funny (h/t Gabriel Rossman)

The article looks at two main things: first, did the rewards affect test scores and other academic outcomes, and second, did creating status groups have effects.

The answer is yes on both accounts. Scores went up significantly on both math and English language arts (ELA) exams. As predicted, effects on other indicators of achievement (grades and scores on exit exams, a different test) were less consistently positive. Moreover, the effects were greater for students near the threshold (i.e. if you just barely missed the gold card one year, your scores increased more). Effects were larger for Asian students than white students, and larger for white students than Hispanic students.

So far, this is consistent (well, with the exception of the racial/ethnic variation) with behavior of the “adolescent econometricians” (p. 266, quoting Manski) assumed by the program. But here’s where it gets interesting.

The authors also use the card cutoffs to perform a regression discontinuity analysis. The students just above the gold card threshold basically look like those just below it. But it turns out they do much better. Or more accurately, kids who got the low-status white card do worse – to the tune of 0.35 SDs on the ELA test, and 0.10 SDs on the math test (both significant at p = 0.01 level).

Interestingly, the impact on grades is even greater: receiving a low-status white card reduces math grades an amount equivalent to moving from a C- down to a D, and ELA grades all the way from a C+ to a D. That’s a big drop. Domina, Penner & Penner speculate that this may be because the status categories are particularly salient for teachers, who may then treat or grade low-status students differently.

So as usual, the lesson here is that while yes, people respond to incentives, they do so in social contexts. You can’t just assume incentives are going to have similar effects on all groups of people, or ignore the effects of new status groups that are produced.

Two other thoughts here. The one interpretation I might quibble with is the authors’ attribution of the racial/ethnic differences to stereotype threat. While that’s one possibility, it’s also certainly plausible that the different response reflects cultural difference (e.g. in the importance attached to test results).

The other is about generalizability. Admirably, the authors don’t really try to generalize at all beyond saying that status categories have effects. But I can imagine people looking at an experiment like this and assuming that whatever the results were, they’d hold up across schools.

But status systems vary from place to place, of course. At the small rural high school I went to, wearing around a card advertising your high test scores would have been a fast ticket to social exclusion. Fryer and Torelli’s “Acting White” paper is hotly debated (and I haven’t followed the whole debate), but they argue that there are racial differences in whether academic achievement is associated with higher social status, and that this gap is greater in schools with more interracial contact. Clearly not every school mirrors the relatively wealthy, high-achieving schools that implemented this program, and not every kid associates high academic achievement with high social status.

That’s just a caution against assuming too much uniformity across social settings, though, not a criticism of the actual paper, which is a great use of a natural (?) experiment. And just think, they didn’t have to suffer through multiple R&Rs to publish it.