orgtheory.net

evaluating causal explanations of samples of one

One of the curiosities of the organization sciences is the prevalence of formal modelling in a world dominated by singular events. After all, aren’t many (most) significant events in organizational life of the ‘one of a kind’ variety? If so, these events are, by definition, resistant to statistical or econometric analysis. Yet we wish to learn from them. Learning about unique events, in turn, often begins with a why-question. Why was Google prepared to pay so much to acquire YouTube? Why did Pfizer fail to secure continued protection of the patent for its blockbuster drug Viagra when challenged in court? Why did Honda succeed so spectacularly in the US motorcycle market?

Explaining why an event occurred typically involves constructing an account of the causes that led to it. These accounts, very roughly, are instances of what we refer to as causal explanations. As the examples suggest, why-questions about unique events and the causal explanations they elicit may reflect important practical concerns – be it for managers or those who study organisations. Where so, there will be premium on getting these explanations right.

How then should the strength of causal explanations of unique events be assessed, given that they cannot be subsumed as members of classes of repeated instances of the kind required in standard statistical analyses? How can those who study organizations assess the relative strength of the causal explanations on offer, including their own? We all know that some explanations are better than others, but what exactly is it for one explanation to be better than another?

Although many of the issues involved are never far from the surface in methodological contributions to the organizations literature (Sutton & Staw, 1995; DiMaggio, 1995; Weick, 1995; Starbuck, 2006; Van de Ven & Johnson, 2006), these questions are rarely raised explicitly. This neglect is doubly surprising in view of the historical importance of single-case research (e.g. Selznick, 1949; Gouldner, 1954; Kanter, 1977, Pettigrew, 1979, 1985; Heimann, 1993; Weick, 1993), and the prominence of case teaching in business schools.

One exception is the article by March, Sproull & Tamaz (1991), from which this blog (and our related paper) stems. Yet, aside from acknowledging the fact that the evaluation of ‘sample of one’ stories is not an arbitrary process, and that there are criteria for differentiating between good and bad stories, the authors leave the gauntlet where it is – and where it seems to remain today.

The preliminaries to a discussion of causal explanations – and their evaluation – are daunting. I won’t test your patience here. A detailed review of the philosophical literature on causal explanation is contained in a recent paper jointly authored with a Cambridge colleague, Jochen Runde (who happens to be one of my more interesting colleagues – now here’s a criterion for choosing your place of work).

Instead, I will ‘test the waters’ by listing our three criteria by which causal explanations of unique events may be rationally assessed. They are deceptively simple (the emphasis being on ‘deceptive’). Essentially, explanations must satisfy three broad criteria: (1) that the factors cited as causes can be accepted as having been present in the run-up to the explanandum event; (2) that those factors can be accepted as having been causally effective in contributing to producing that event; and (3) that, given an affirmative answer to (1) and (2), the causes actually cited in the explanation explain well, taking into account various contextual and epistemic considerations relating to the intended audience for the explanation, and the interests and theoretical presuppositions of the person providing the explanation.

No doubt our argument will be challenged on the grounds that the criteria we have presented are very general and (i) will therefore not have sufficient bite to discriminate between competing explanations in practical situations, and (ii) are not up to guaranteeing what John Kay calls the ‘ultimate truth’ about episodes as complex as the well-known “Honda effect”. On the first objection, we have pitched our account at a highly abstract level, and realize that there may be more specific methods and techniques in particular areas of research that can be used to distinguish between competing explanations. But the existence of such methods would not undermine our general account – nor need of it. On the second objection, our never being able to achieve the ‘ultimate truth’ about the causes of any event seems to us an inescapable part of the human condition. This is something that would require a God’s eye view of the full list of causes of that event. Given that this is likely to remain beyond reach, even our very best explanations will invariably be partial and interest-dependent (and it is worth remembering that even the very best statistical analyses are no better on this count). The best we can hope for is that weaknesses or biases in causal explanations can sometimes be identified and that weak explanations can be ruled out.

For a more detailed discussion, please see: Runde, J. & de Rond, M. ‘Evaluating causal explanations of samples of one’, Working Paper. (A draft of this paper can be emailed upon request).

Written by markderond

June 16, 2008 at 7:41 pm

Posted in uncategorized

8 Responses

Subscribe to comments with RSS.

  1. I have not read the paper in question, so take my response with a grain of salt. I suppose that I take slight issue with the premise of the argument. I often think that I am setting out to explain a particular event, but in actuality what I usually accomplish is demonstrating that the event in question is an example of a larger class of events although the event in question may modify the scope conditions originally established. At what point is an event truly unique? When a theoretical explanation of it is simply an abstracted description of the event? Further, what role does randomness play?

    Like

    trey1

    June 16, 2008 at 8:03 pm

  2. […] summer at one of my favorite blogs.  Here’s an excerpt from an insightful, efficient post on learning from one-time organizational events: One of the curiosities of the organization sciences is the prevalence of formal modelling in a […]

    Like

  3. Nice post. A few quick thoughts:

    Your intuition also resonates with the recent call to explain “extreme events” in the organizational sciences (McKelvey and others). While much of organizational activity indeed is routine, repetitive etc, the more interesting of it all is (potentially) in the extremes and the exceptions; in particular from the perspective of understanding org heterogeneity (performance or otherwise).

    One feasible solution (alternative explanation), though, to explaining extremes/rare events/samples of one, might simply be random variation (variation-selection-retention framework, if you will). As Alchian (1950, JPE) notes — citing the mathematician Borel’s thought experiment — if two million Parisians flipped coins we’d get heterogeneity and rare events galore, but it’d simply emerge via chance.

    Like

    tf

    June 16, 2008 at 9:46 pm

  4. Mark, Your post reminds me of one of the few classroom moments I remember from grad school. Our econ history teacher, after going through all the different competing explanations for why the industrial revolution in the west occurred first in Great Britain and not elsewhere, said, “Well, ultimately, we only have one data point, and lots of theories are consistent with one data point.”

    Like

    mikemcbride

    June 16, 2008 at 10:17 pm

  5. This discussion also makes me think of Nassim Nicholas Taleb’s work (Fooled by Randomness, The Black Swan).

    Like

    trey1

    June 16, 2008 at 11:50 pm

  6. […] fabio, philosophy by fabiorojas on June 17th, 2008 Mark wrote a very interesting post on the study of rare cases. Let me the take the opportunity to offer my own take on a related topic, what I call the […]

    Like

  7. […] 17, 2008 I am referring now to a discussion in OrgTheory , about, among other things, whether it is better to study extreme (or rare) cases, or to study the […]

    Like

  8. Obtaining a casual explanation — i.e., a proof — of an individual event is in some ways no different from solving an individual problem in geometry. Given a general category of events, such as business, geometry, or drilling for oil, one must either have or find the fundamental factors that CAUSE events in the category to turn out the way they do. A statistical review of a category can provide fairly reliable “fundamentals,” the factors that determine the outcome of events in the category, but in general, only the techniques of logical deduction and proof (itself a category) can reliably provide a category’s fundamentals. Until a category’s fundamentals are known, it is impossible to obtain either a causal explanation for, or a true understanding of, any event within the category. Once the fundamentals are known, causal explanations of individual events are very reliable, limited only by our knowledge of the category to which the event belongs, and the basic errors of logic and reasoning that can occur in any analysis.

    However, the authors face a major problem, which is that the modern scientific method is not really compatible with causal explanations. Renaissance scientists like Newton, Galileo, and Gauss believed that there is an inherent, but largely hidden, structure of nature that causes events to turn out the way they do. For them, science was the unraveling of a mystery, of using the techniques of logic and reason to reveal the hidden structure, i.e. fundamentals, of some category (see “Reinterpreting Galileo” edited by William Wallace, or even Galileo’s own “Two New Sciences”). The philosophers of modern science like Ernst Mach, however, came to the conclusion that this “hidden structure of nature” was essentially mythical, and they successfully shifted science away from theories that describe some mythical structure of nature, to theories that “predict the results of events.”

    This sounds like a real improvement, and in a debate setting, it is indeed much easier to defend a theory that merely predicts results, than to defend a theory that purports to describe some hidden, invisible structure of nature. However, ANY claim of cause and effect, whether in business, geometry, or physics, assumes that something does exist which forces a certain result to occur. It is not an exaggeration to say that unless we accept the premise that a structure of nature exists which causes events to turn out the way they do, that the techniques of proof and understanding are not available.

    Plato used a single system of thought for both debate and science. It was his star student, Aristotle, who concluded that we must have two systems of thought, one for debate, politics, and the social arena, and another for science, truth, and intelligence. He analyzed the elements of human thought (grammar), and created logic, the science of reliably deducing facts that cannot be seen, from facts which can be seen, thereby making it possible to reliably describe the largely hidden structure of nature which causes the events around us to turn out the way they do, events previously believed to be the whim of the gods. As Ayn Rand pointed out, the modern scientific method is in fact a return to Plato’s “one mind fits all” system of thought and reasoning, a “dialectic” scientific method, in contrast with Aristotle’s true scientific method, or “reality” scientific method, and until that fact is recognized and corrected, the authors’ attempt to have the scientific community accept causal explanations of individual events is probably doomed.

    Like

    Philip Yates

    July 30, 2008 at 6:50 pm


Comments are closed.

%d bloggers like this: