the scientific method fallacy
Lieberson and Horwich’s article on implication (profiled yesterday) raised an issue that I think is very important in social research. I call it the “scientific method fallacy.” Here’s how I would explain it:
The scientific method fallacy is when a researcher mistakenly assumes that a tool used by physical scientists is the only legitimate way to do research. In other words, they move from “X is a great tool for science” to “X is the only way to do science.”
Examples of the scientific method fallacy: “Experiments are the only way you can really know anything.” “You really don’t have a clear theory unless it is expressed mathematically.”
The underlying philosophical claim is that science is pragmatic. The world is too complex and hard to be captured by any single tool, so we need multiple tools. A formal model, or an ethnographic observation, is a map of the world, not the world itself, which suggests the needs for more kinds of maps.
If you actually look at what scientists do, you see that no single method rules, even though some are clearly more popular than others. Instead of saying that “real scientists use X,” one should read science journals. A medical journal might include randomized controlled trials, qualitative case studies, observational data, and even opinion pieces. In engineering, it is common to find reports on prototypes. You can learn a lot from building something, even if they theory isn’t nice and neat. There’s even “grounded theory” in the physical sciences from time to time. For example, when the first particle colliders were invented, physicists had a great time just seeing what new particles were made and how that might lead to new theory. And of course, you also see lots of formal models and controlled experiments across scientific areas.
The bottom line is that reality is more complicated than suggested by those pushing the scientific method fallacy. Real science is messy and that means that science progresses on multiple fronts. If an experiment can be done to convincingly settle an issue, great. But a lot of times, it’s not possible, or even desirable. The lesson for social scientists is that we should stop listening to those who say “real science is done this way” and instead have the courage to make science’s many tools work for us. And that’s the way real science works.