what are school effects?
A few days ago, I asked if readers believed in school effects. The poll suggested that most orgtheory respondents do believe that schooling makes a difference. In the comments, many people asked: what counts as a school effect? joshtk76 noted that different features of schools seem to have different effects on students.
Ok, let’s sort this out. First, in the sociology and education literature on schools, there is a statistical definition of a “school effect.” It simply means that if we assign a variable for a type of school (e.g., public vs. private) or specific school (Indiana University), then there will be an effect on the dependent variable of choice (e.g., income or learning measurement). A lot of classic studies in sociology and economics then play the game of making the effects go away. For example, Coleman’s study used family background to make school effects disappear. Card and Krueger try to measure student aptitude to make college specific effects disappear. Often, school specific effects are reduced or go away entirely. The flavor of many studies are like that – add family, or cognitive ability, or whatever and school effects diminish.
Now, as the commenters note, there is still some ambiguity. There is stuff that is school specific that is not “school,” such as your friends at school. Luckily, the literature does give some guidance. For example, we do know that there are teacher effects. There seem some teachers who are really good at teaching. If a school has good teachers, then students will learn more. On social capital, the evidence is mixed. If you read Muow’s work on networks and jobs, the evidence is mixed, which suggests that social capital from schools doesn’t have that big of an impact.
If the evidence is so mixed, I wouldn’t put a whole lot on school effects unless there is a strong reason otherwise. For example, elite occupations (e.g. academia) seem to revolve around certain elite schools. Another example: very low SES students, the one group where we consistently find school effects. So when thinking about school effects, your presumption should be “not much, but I am willing to consider the evidence for specific groups of people.”
Inquiring Minds Want to Know: From Black Power/Grad Skool Rulz
Altonji and Mansfield have a nifty paper in the _Whither Opportunity_ book that came out last year. They reproduce the usual finding that school differences explain a low proportion of the variance in future wages, but then they look at outcomes for schools at the 10th percentile and 90th percentile of “school quality” (predicted outcome based on school characteristics + school residual) and find substantial gulfs. The point being, don’t sneer at school effects!
LikeLike
joshtk76
December 13, 2012 at 5:01 am
joshtk76: Thanks for the citation. I’ll definitely take a look. Your description seems to reinforce my major point. If you have to be a few standard deviations above/below the mean, then I’ll stick by my line that school effects are hard to find!
Maybe there’s a deeper lesson. There are school effects but most school districts/administrators are unable to build a school with the right characteristics.
LikeLike
fabiorojas
December 13, 2012 at 6:11 pm
Link:
Click to access Altonji%20and%20Mansfield%20(2010).pdf
LikeLike
joshtk76
December 13, 2012 at 8:37 pm
Really – every fifth school (at least apparantly) and it’s “hard to find”? This sounds like a typical case where people have tried to find an “average” effect, while the effect is clearly contingent on some non-average conditions.
LikeLike
Anonymous
December 13, 2012 at 9:33 pm
Anon: good point. From a policy perspective, you have to go through 9 schools before finding a good one. What if we said that only one in ten doctors actually did any good?
LikeLike
fabiorojas
December 14, 2012 at 1:06 am
“Anon: good point. From a policy perspective, you have to go through 9 schools before finding a good one. What if we said that only one in ten doctors actually did any good?”
What if we said that a pill for HIV works for one in ten patients?
does this pill have an effect?
LikeLike
cvillage
December 14, 2012 at 3:37 am
Out of 10 schools, you have to look out for 2 school. Most people probably look at +10 schools I imagine, so are likely to encounter 1 of each. If you improve your ability to select the right school by 30-40%, then this suddenly becomes very important.
About the doctors – 8 are fine and average, 1 sucks, 1 is excellent. Again, the choice becomes important. At least easier to switch doctors…
LikeLike
Anonymous
December 14, 2012 at 8:45 am
Cvillage: I was once in a simar situation. My mother was diagnosed witha terminal illness. The docs tried a treatment with those odds. Even then we agree that it wasn’t evidence of effectiveness.
Anon: if I understood the original comment, the top and bottom deciles have an effect. I assume they auve different effects – top schools boost outcomes and low schools make you worse. Yes? If so, there is an effect of 1 in 5 schools, but a good effect of 1 in 10 schools. Not thrilling from a policy view, especially given that schooling is very expensive.
LikeLike
fabiorojas
December 14, 2012 at 2:24 pm
Yes, but I thought we had a “is it important to pick the right school effect”, and the fact seems to be that 1 out of 5 choices are significant. And it does not, again, seem unlikely that a choice is completely random. Chances of making a valuable (meaning that you avoid the worst situation to get to average and/or go from average to good) decision is fair.
In regression analysis we consider very low R squares to be of interest – in this case it’s at least 1/5th of the cases that are of interest. i find it difficult to argue that this is not of interest. Life is about making tough decisions and every fifth kid seem to be significantly affected by school choice (all assuming the study is correct).
LikeLike
Anonymous
December 14, 2012 at 4:16 pm
Anon, once again. You make fair points. Let me read the cited study and I’ll see if I warm up to your position. Thanks for posting.
LikeLike
fabiorojas
December 14, 2012 at 6:50 pm