popper and the platypus
Yesterday, I wrote about the importance of case studies and general therories, and how they are really two sides of the same coin. In a comment, Mike asks:
I like the argument you’re making, and I am highly supportive of and interested in the study of singular or rare cases. Yet, I wonder if a Popperian might argue that studies of singular events are more difficult to falsify and thus outside the realm of science at worst or on the fringe at best.
Once again, let’s apply this sort of Popperian logic to actual science. If you can’t study singular cases, then the following studies are fringe science:
- You can’t study the platypus because there is no other mammal like it.
- You can’t study the human brain, because it is the only one capable of higher abstract functions like counting and language.
- You can’t study integer numbers, or even fractions, because most numbers are actually irrational.
- You can’t study the planet earth, because it’s the only one, so far, that appears to have life on it. Heck, for a long time, it was the only planet we knew about!
Once again, the reasoning is absurd because the premise is false. The deeper lesson is that facts don’t exist by themselves. Facts usually exist in relation to theories that predict certain kinds of distributions. Ignoring cases leads to bad conclusions because you toss out information about what the distribution is really like. Multiple studies of odd cases can clarify what those limits are. Theories also have assumptions, and cases can reveal the problems with the assumptions.
For example, one biological prediction is that “all mammals give live birth.” If we acted in a thoughtless statistical fashion (a la population fallacy), then we’d either simply not record the platypus as a data point because it’s a case study, or simply code it into a variable. Then we’d look at the population average and conclude “all mammals give live birth,” even when it isn’ true. Another example, if we found life on a single other planet (so far, Earth = n =1!!), it would greatly change our theories. According to the anti-case study people, the NASA scientists studying Mars are fools because even if they found life, the sample size would only be N=2! The point of studying Mars is that if we found life, it would help us understand the limits of earth focused biology, even if we couldn’t immediately generalize to a more comprehensive theory of life.
I’ll conclude by offering another hypothesis: the anti-case study bias is driven by mathematism, the wrong belief that math is an essential quality of real science. Here’s the real truth – math is sometimes used to refine and verify ideas created through observation and thought experiment. However, real science comes from observation, thought experiments, and critique. Statistics can tell you about trends, but only observation and reason can tell you about the underlying structure and quality of your theory – and that’s where case studies come in.
I recall Andrew Gelman making something like this point from the other side in response to a question from someone who had more data points than they could manage:
Kieran
June 18, 2008 at 1:27 pm
I am not a strict Popperian, so I cannot speak with authority here, but I am not sure that your platypus starting point is fine enough. Some theories about the platypus might be falsifiable, e.g., a scientist could experiment with multiple platypi and test specific hypotheses by generating scores of data.
In the end, I think the hypothetical Popperian’s issue might be to what extent falsifiable theories can be made concerning rare cases. Some claims are falsifiable while others are not. It is the nature of those claims that makes them science or not science. The ones that cannot be falsified would not be science per se (or they would be on the fringe). But the hypothetical Popperion could still be interested or even persuaded by such work. Eg, Popper himself was a philosopher of science who argued in favor of democracy though those views are not exactly falsifiable.
If there’s a good Popperian out there, please jump in!
mikemcbride
June 18, 2008 at 4:53 pm
I’m a bit a of Popperian (not sure if I’m a good one; let’s see).
Singular events are not just “difficult” to falsify, they aren’t really what falsification was about. Falsification was about theories, i.e., general claims (preferably universal ones).
Popper wasn’t talking about what you can or cannot “study”, he was talking about the scientific basis of theories. As long as we agree that a field study of a single platypus does not give as a basis to generalize about life on earth, there is no problem. Popper would say you can “study” whatever you like.
But if you’re going to say something like “All mammals give birth to live offspring” (or not) then you’re going to need more than a case study. The platypus could be used to defend the claim that “Not all mammals give birth to live offspring.” But how interesting is that? It’s like saying, “Some mammals have two legs.” A case study could determine that, but what have we learned?
I think it is very difficult to say what finding life on Mars would teach us about life in general. I don’t think we’re testing any of our biological theories on Mars today. I think we’re just curious about Mars. The scientific mission there is a bit questionable. But the adventure is beyond dispute.
Thomas
June 18, 2008 at 5:23 pm
I.e., I don’t think there is a Popperian argument against doing case studies. There is a Popperian arguments against using them as the basis of the generalisations. And that argument, it seems to me, holds. But I don’t demand generalisations from all inquiries.
Thomas
June 18, 2008 at 5:37 pm
It’s worth remembering that Popper’s ideas about falsifiability aren’t successful in their goal of deciding whether this or that study or theory should count as scientific, or even in the weaker aim of deciding what is the right sort of attitude to take to dealing with the relationship between theory and evidence.
Kieran
June 18, 2008 at 7:03 pm
Kieran is right, of course. As a general theory of scientific inquiry, critical rationalism is hardly the end all and be all. But Popper’s lack of success is really just part of the (general!) failure of 20th C philosophy of science. That is, I’d like to know what you count as a success before we make too much of Popper’s failure.
I used to dismiss Popper as, I guess, “falsified”. But today I find falsificationism to be one of the most useful heuristics for making sense of scientific claims: what is X saying will NOT happen on the basis of the theory X is defending?
And that’s really the difficulty with a straight case study. What do the “facts of the case” tell us will not, cannot, or should not happen? Often very little.
Thomas
June 18, 2008 at 7:17 pm
Thomas, I think you are mistaken that cases can’t create general theories.
Let’s get back to the platypus. Before the discovery of the platypus, Western science said “being a mammal is inconsistent with egg laying.” After it’s discovery, the new general statement is “being a mammal is consistent with either live birth or egg laying.”
Seems like a generalization to me and it was based on a single species.
Fabio Rojas
June 18, 2008 at 7:23 pm
It’s true that, while given up as a bad job by philosophers in the 1970s or thereabouts, the demarcation problem has returned as a political matter with the rise of Intelligent Design types, and so on.
Kieran
June 18, 2008 at 7:30 pm
I don’t deny that. What I am saying (and this is pretty orthodox Popperianism) is that its bad science just to include the case by adding “or …” and then a description of the exception. What we need is a new, “bolder”, generalisation that captures the reproductive processes of mammals.
Popper would find the “or egg laying” dubious because it suggests an “ad hoc” way of getting around evidence to contrary. What you are really sayign is “live birth or egg laying or …” whatever might turn up. Well that’s not really a theory. That’s just a strategy for accomodating any observation.
Falsificationism forces us to say “Eggs?” “Really?” “Are you sure?” “A mammal?” and then find some way to make sense of it.
Thomas
June 18, 2008 at 7:38 pm
I just had quick look at Wikipedia…
It seems mammals are not distinguished by reproductive characteristics today. Probably because “live birth” was falsified. It looks like “sweat glands” is the clincher. The other is: “two knobs at the base of the skull which fit into the topmost neck vertebra” (which has somethign to do with hearing, I think.)
Suppose someone explained Google’s dominance by an incentive system not used in any other industry’s dominant firms? Wouldn’t we prefer something else?
Thomas
June 18, 2008 at 7:51 pm
Where does Weberian ideal-type analysis fit here? That strategy uses a case which might not be merely statistically rare but which might not exist at all.
JayLivingston
June 18, 2008 at 8:16 pm
Thomas – The point of the platypus is to show that assumptions are wrong. Thus, case studies are of extreme importance, even goofy ones like the platypus. Exclusive large N thinking allows us to evade troubling cases that show flaws in assumptions.
It is a separate question as to what the next step can be. Using a case study to motivate an ad hoc amendment is logically valid but highly uninformative. A better move is to dig deeper and think about how platypuses and other mammals fit together, even if they differ in one very important anatomical feature.
But let me emphasize – we wouldn’t even know the relative importance of reproduction in biological classifications if we didn’t take the time to do case studies. And it is a different question how the puzzles raised by case studies are to be resolved.
Jay – I don’t have the energy to fight Weberians after cleaning up the Popperians…
Fabio Rojas
June 18, 2008 at 9:11 pm
I don’t think you and Popper would disagree here. (Nor do you and I.) In fact, your beef seems to be more with “verificationists” (logical positivists), at least before they too gave up on induction. That’s where all the big N stuff might have counted for soe
mething. Falsificationism makes the single case interesting in precisely the sense you need, since just one platypus will do the trick of forcing us to rethink our categories.
If you weren’t a falsificationist, the single case couldn’t challenge assumptions.
I don’t know what his position on statistics was specifically, but I’ve never connected Popperianism with large data sets. Given a good theory, you need only a few intelligently chosen observations for the purposes of testing.
Then again, maybe this is just another classic case of “At least Popper wasn’t a Popperian!” (You may know some “Popperians” that really do need to be corrected on this point.)
Thomas
June 18, 2008 at 9:21 pm
Thomas – I think the issue that’s at stake is the contribution of cases, especially unusual cases. A lot of working social scientists look down on case studies as fringe science or just unscientific. A Popperian view provides at least one justification – cases that challenge predictions or assumptions about is possible. There are other justifications – cases can help generate hypotheses, add to the total stock of knowledge, or act as laboratories for certain processes. But people resist all these as unscientific because they don’t easily fit into large N reasoning, and thus dump cases.
Fabio Rojas
June 18, 2008 at 9:58 pm
If we’re willing to entertain a shift from Popper to Lakatos for a minute, Emigh 1997 deals with some of these “platypus issues” regarding the ways in which single negative cases contribute to theory development.
(I tried last night to post a suggestion that debates in comparative politics might provide a few interesting ideas on standards for causal inference in case studies; but the comment was lost to the ether. But since I’m back, I was thinking, for instance, of Gerring’s “What is a Case Study and What is it Good For?”; George and Bennett’s Case Studies and Theory Development in the Social Sciences; as well as Collier and Mahoney 1996.)
robjansen
June 18, 2008 at 10:03 pm
Yes, I think Popper is best used to justify particular kinds of case studies. My concern was with the idea that “You can’t study the platypus because there is no other mammal like it” somehow expresses a “Popperian logic”. I don’t think it does at all.
What Popper was also right about, however, and you may disagree here, is that there is no such thing as simply “adding to the total stock of knowledge”. The specifics of any given case will be interesting (to science) in so far as a theory makes predictions about it. It is unscientific to investigate something in detail just out of curiosity and with the aim of compiling a list of true sentences about it.
One way to make up for such unscientific research is generate hypotheses, as you say. But these become scientific not from their basis in the case, but from their further testing.
Science happens when we approach phenomena as specific cases OF more general truths.
That said, I don’t think organization studies should be “scientific” every second of the day.
Thomas
June 18, 2008 at 10:22 pm
Rob Jansen’s comment died, so I post it here:
If we’re willing to entertain a shift from Popper to Lakatos for a minute, Emigh 1997 deals with some of these “platypus issues” regarding the ways in which single negative cases contribute to theory development.
(I tried last night to post a suggestion that debates in comparative politics might provide a few interesting ideas on standards for causal inference in case studies; but the comment was lost to the ether. But since I’m back, I was thinking, for instance, of Gerring’s “What is a Case Study and What is it Good For?”; George and Bennett’s Case Studies and Theory Development in the Social Sciences; as well as Collier and Mahoney 1996.)
Fabio Rojas
June 19, 2008 at 12:09 am
[...] one population at a time. These examples are important in light of the excitement generated during earlier discussions on this blog about studying unique or extreme cases. While I wholeheartedly agree with [...]
the consequences of selective sampling « orgtheory.net
June 25, 2008 at 6:09 am