karl popper vs. the identification people

Truth be told, I think Popper would have approved of the identification effort. How could the person who advanced falsification not have approved of an attempt to address endogeneity and unobserved variables? But most people ignore an important aspect of Popper’s beliefs about scientific theories. He also thought that theories deserve special attention if they make unexpected predictions. Thus, falsification was very powerful if combined with creative derivation of hypotheses.

Now, identification is not in any way contrary to that view. Instrumental variables, or any other technique addressing identification, can be used to test  unexpected hypotheses. However, real scientific practice is about trade offs. Should I spend my time on theory building or some other activities? My view of the identification craze isn’t that it’s wrong. My view is that it’s “end game.” In other words, identification is a luxury when you have an abundance of data and a pretty clear idea about what causal effects you care about. It’s the epitome of normal science.

That’s why I keep a little distance from the identification craze. Sure, I’ll agree it’s important, but too much identification means that you are unwilling to do difficult theory work that’s needed for real scientific progress. You’ll reject research questions that don’t have some built in identitifcation strategy. Any time people focus on tools, they tend to work less on coming up with novel predictions and they work on adjudicating between the same old competitors.


Written by fabiorojas

October 21, 2009 at 3:16 am

9 Responses

Subscribe to comments with RSS.

  1. Excuse my ignorance, but what is the identification effort?



    October 21, 2009 at 2:56 pm

  2. Josh: There’s been a resurgence in interest in how one may address causality through natural experiments, instrumental variables, discontinuity regression, and the like.



    October 21, 2009 at 3:18 pm

  3. Pierre Azoulay wrote a great set of posts on the identification movement earlier this year (here, here, and here).



    October 21, 2009 at 3:56 pm

  4. Fabio, at the same time, don’t you think that methods that are well-suited for testing causal claims can be theory’s best friend? Seems to me theoretical progress happens via a conversation between theories and data. In the absence of strong methods for sorting out which works best, and where they apply, creative, well-constructed, clear theories can just proliferate. Without the input of strong, convincing methods (be they associated with the identification movement or not), the standards for evaluating theories drift from validity to aesthetics, elegance, cleverness, use of fancy language, etc.


    Thorstein Veblen

    October 21, 2009 at 4:30 pm

  5. Thorstein: I actually agree with you in principle – theory and empirics need to be together. I also agree that we need methods specialists. What I am trying to get at is that there is a temptation to dump theory for ever more sophisticated hypothesis testing.

    The other thing that distresses me is that we’re seeing fewer and fewer big theory developments, and more “middle range.” I’ve argued in the past that many social science frameworks date to the 1970s or so. My suspicion is that the energy that used to go into theory building is now going into data analysis. My gut is that we need to pull back and so some deep thinking and not obsess over the latest natural experiment.



    October 21, 2009 at 4:38 pm

  6. […] Fabio Rojas says what I wanted to say, rather more concisely. He notes that “identification is a luxury when […]


  7. […] (What would Popper Do? Or perhaps more accurately, How would Popper feel?) An interesting thread on “identification” over at […]


    WWPD? « Permutations

    October 21, 2009 at 7:35 pm

  8. I like the idea of casual effects. :)


    Stuart Buck

    October 22, 2009 at 4:15 pm

  9. Is it reasonable to assume that in the social sciences the methods and means to test theory usually lag far behind the initial theory development?

    If so, is it reasonable to assume that what we are seeing is not really the fact that people have dumped theory, but rather, that the methods are making up for lost time?

    And if that’s true, could we then see a correction once all the low hanging fruit have been picked?


    Robert Vesco

    October 24, 2009 at 3:58 am

Comments are closed.

%d bloggers like this: