orgtheory.net

interaction models

Omar

Deep within the recesses of anybody who has gone to grad school in a PhD program in social science, there exists a small neural net that stores information about interaction effects and interaction models. Interaction effects are cool because they allow us to make nice counter-intuitive hypotheses, or hypotheses that serve to bring together warring swaths of literature with the conciliatory proposition that “it depends.”

So the standard “interaction” hypothesis is:

hypothesis 1: X has an effect on Y, when condition Z is within a certain range of values (let’s say Z=0) and does not have an effect (or has the opposite effect) when condition Z is within a different range of values (let’s say Z=1).

Thus in contrast to the unconditional model, which is written:

Y=a+b1X+b2Z+e

The interaction hypothesis can be tested by specifying:

Y=a+b1X+b2Z+b3XZ+e

A while back I blogged about some really fascinating political science research on symbolic racism and the Republican Southern strategy. It showed that when Y=vote republican, X=negative racial attitudes toward blacks and Z=lives in the south, you find a positive interaction effect for the more recent period (1980s and 1990s) in contrast to its null effect in the 1960s and 1970s.

So we are agreed interaction models are awesome. However, as your stats 101 teacher told you, you have to be careful about two things: (1) never omit the main effects. Thus you don’t test hypothesis 1 using any of these specifications:

Y=a+b1X+b2XZ+e

Y=a+b1Z+b2XZ+e

or Alanis forbid:

Y=a+b1XZ+e

And (2) when interpreting b1 and b2 in the fully specified model, remember that those effects are conditional on the value of the other variables. b1 is now the effect of X on Y when Z=0 and b2 is now the effect of Z on Y when X=0. If your variables don’t have a meaningful zero point (like a racial attitudes scale), center them at their mean so that you can say “b2 is the effect of being Southern on voting republican for those who have average levels of racial animus towards blacks.”

Seems simple. Everybody knows this. Why am I even explaining this to you? Well, as noted by Branbor, Clark and Golder (2006) in a recent article in Political Analysis, a survey of 156 articles published in the major Political Science journals shows that only 10% of researchers specified their interaction models correctly. A large chunk of them outright omitted main effects, which can lead to incorrect significance tests of the interaction term. In some of these articles the entire contribution was riding on the interaction term. So things are not so simple. Consider the horror:

In an award-winning article in the American Political Science Review, Boix (1999) examines the factors that determine electoral system choice in advanced democracies. He makes two main conclusions. First, ethnic or religious fragmentation encourages the adoption of proportional representation in small and medium-sized countries (621). He draws this conclusion based on a model that includes an interaction term between ethnoreligious fragmentation and country size. However, he does not include either of the constitutive terms. When these terms are included, there is no longer any evidence that ethno-religious fragmentation ever affects the adoption of proportional representation (italics added).

You should read the article to see other horror stories. The lesson: if your dissertation/paper is riding on an interaction effect, don’t be a fool. Estimate a fully specified model.

Written by Omar

October 20, 2007 at 11:26 pm

6 Responses

1. In general I agree that one should include the main effects in a model. However, it is not in principle wrong to only estimate interaction effects. If one’s theory strongly suggests that an interaction should be the case and not a main effect, then just including an interaction term would be appropriate (and one could in fact obtain misleading results if a main effects were added to the model).

Like

anonymous

October 21, 2007 at 8:08 pm

2. anonymous,

in such a case you’re failing to make conservative assumptions. the best example of this i can think of is a 1982 ASR article by James Davis where he tested the “nouveau riche” hypothesis (which is basically an argument about the interaction effect of origin and destination class on attitudes) and found it to be entirely explicable by the main effects. if you had a theory of the nouveau riche that strongly suggested they would be more conservative than old money, you wouldn’t include the main effects and you would spuriously find your theory confirmed.

(myself, i’m skittish about interaction effects because i worry about the interpretation, especially colinearity)

Like

gabrielrossman

October 21, 2007 at 10:42 pm

3. anon: I can’t think of any theory in sociology that offers hypotheses at a level of specificity that allows for any researcher to make that claim. Besides, the hypothesis that B1 and B2 are zero is testable, so we don’t have to leave it to armchair speculation. All you have to do is estimate the fully specified model and do an F (or likelihood ratio) test. The problem with some of the papers that are mentioned in the article is that they included an interaction term without main effects without even checking and published the results. When Brambor et al reanalyzed the data with the main effects included they found that the interaction term was no longer significant, which invalidated the main conclusion of various papers.

Gabriel, I wouldn’t worry too much about interpretation and/or collinearity. Friedrich (1982) showed a while ago that both of those concerns tend to be highly overstated in the literature. For those who are interested, Brambor keeps a nice page at NYU with examples (and Stata code) on how to deal with most common issues with interaction models.

Like

Omar

October 21, 2007 at 10:59 pm

4. Here is a more subversive version of Omar’s original post:

If the coefficient of an interaction effect is substantial, net of the main effects, then there are really only two possible things going on: the main effects have not been scaled properly according to one’s theory and/or there is a meaningful omitted variable that, if it were included in the model, would explain away the interaction effect. Thus, the only good model is a model in which there are no interaction effects of substance.

This subversive line of argument is (if my memory is correct) attributable to Bob Hauser’s 1970s critique of Raymond Boudon’s book that rested on an interaction effect. But, really, it is a point straight out of statistics.

All that being said, heterogeneity of causal effects is of the utmost importance substantively, and regression is a poor framework for revealing all of it. The Angrist and Heckman work in econometrics over the past decade is devastating on this point.

Like

Steve Morgan

October 22, 2007 at 2:45 pm

5. […] by fabiorojas on April 14th, 2008 Previous orgtheory posts on  controversial statistical topics: interaction terms, Bayesian statistics, p-values and survey response […]

Like

6. Hi
there is one exception of omitting the constituent term (main) effects in the multiplicative interaction model as suggested by the influential paper of Brambor (2006). If the model involves a variable with natural zero then in that case omitting the constituent term does not lead to any errors (See Page 68, 2nd paragrah)

Like

Waeed

June 7, 2009 at 5:27 pm