let’s hear it for null results

A common, and important, critique of journals is that they don’t want to publish null results. So when I saw a new piece in Socio-Economic Review yesterday reporting essentially null findings, I thought it was worth a shout-out. The article, by economist Stefan Thewissen, is titled, “Is It the Income Distribution or Redistribution That Affects Growth?” (paywalled; email me for a copy). Here’s the abstract:

This study addresses the central question in political economy how the objectives of attaining economic growth and restricting income inequality are related. Thus far few studies explicitly distinguish between effects of income inequality as such and effects of redistributing public interventions to equalize incomes on economic growth. In fact, most studies rely on data that do not make this distinction properly and in which top-coding is applied so that enrichment at the top end of the distribution is not adequately captured. This study aims to contribute using a pooled time-series cross-section design covering 29 countries, using OECD, LIS, and World Top Income data. No robust association between inequality and growth or redistribution and growth is found. Yet there are signs for a positive association between top incomes and growth, although the coefficient is small and a causal interpretation does not seem to be warranted.

Okay, so there’s the “signs for a positive association” caveat. But “the coefficient is small and a causal interpretation does not seem to be warranted” seems pretty close to null to me.

In light of the attention this report from S&P has been getting — e.g. from Krugman today (h/t Dan H.) — all solid findings, null and otherwise, on the inequality-growth relationship warrant publication. Hats off to SER for publishing Thewissen’s.


Written by epopp

August 8, 2014 at 4:35 pm

One Response

Subscribe to comments with RSS.

  1. Thanks for this comment, Elizabeth, and the cheers for Socio-Economic Review. It is certainly an important function of social science to debunk ideological thinking, and I think the equity-efficiency trade-off is a good target for this. I think that this is an important and underestimated role of how our work can potentially have “impact” (and in a different way than debates here on the blog around the “impact factor”).

    A more philosophy of science view on the null results issue concerns the role of falsification in the social sciences. What do we do with null results? My impression is that falsification remains somewhat neglected for various reasons that are interesting to discuss. Sociology is often considerd a “low paradigm” field in that we lack a unified set of concepts and theories that define the state of knowledge. However, one reason for this is quite possibly a self-reinforcing dynamic: to the extent that we are resistent toward “rejecting” theories. they just proliferate. One factor at play is probably the unique problems that social sciences have in terms of the access and quality of data, and consequently weak state of measurement. Rather than decisively test and reject theories, many authors figure that their data are too limited and just keep going past negative results to find the (usually more marginal) positive ones. These positive results then become ever more contextualized with mediator and moderator variables, where new studies “save” theories by finding small effects under certain boundary conditions. If this is indeed true, the way we are using empirical data probably leads to the proliferation of new concepts and approaches at the expense of cumulative understanding.

    I think an alternative must be a more confident and self-reflected form of case-based and historical social science. Here I think that null results and debunking generalizations play an important role, and we certainly need lots of contextualization and boundary conditions. But I would shift the accent here in that should focus more attention on those deviant cases (e.g. countries that have high growth and high equality in the terms of the SER article) as phenomenon-driven opportunities for new theory. But this is another subject for another posting!

    Liked by 1 person

    Gregory Jackson

    August 11, 2014 at 9:49 am

Comments are closed.

%d bloggers like this: