orgtheory.net

problems vs. theory

What makes a study interesting?  Is it the empirical phenomena that we study or is it the theoretical contribution? For those of who are really paying attention (and I applaud you if you are), you’ll notice that I’ve asked this question before. It’s become a sort of obsession of mine.  For the field of organizational theory, it’s an important discussion to have, although it’s not one that will likely yield any consensus. Scholars tend to have very strong opinions about this. Some people feel that as a field we’ve fetishized theory to the point of making our research inapplicable to the bigger world we live in. Others claim that by making “theoretical contribution” such a key component of any paper’s value, we ignore really important empirical problems.  But in contrast, some scholars maintain that what makes our field lively and essential is that we are linked to one another (and across generations) via a stream of ideas that constitute theory. What makes an empirical problem worthy of study is that it can be boiled down to a crucial theoretical problem that makes it generalizable to a class of phenomena and puzzles.

At this year’s Academy of Management meetings, I was involved in a couple of panels where this issue came up. It was posed as a question, should we be interested in problems or theory?  If we are interested in studying problems, we shouldn’t let theoretical trends bog us down. We should just study whatever real world problems are most compelling to us. If we’re interested primarily in theory, we need to let theory deductively guide us to those problems that help us solve a particular theoretical puzzle. Some very senior scholars in the field threw their weight behind the former view. I don’t want to name any names here, but one of the scholars who suggested we should be more interested in real-world problems is now the editor of a major journal of our field. He offered several examples of papers recently published in that journal that were primarily driven by interesting observations about empirical phenomena.

One of the new assistant professors in the crowd threw a pointed objection to the editor. And I paraphrase, “This all sounds great. I’d love to study empirical problems, but reviewers won’t let me! They keep asking me to identify the theoretical gap I’m addressing. They demand that I make a theoretical contribution.” Good point young scholar. Reviewers do that a lot. We’ve had it drilled into us from our grad school days that this is what makes a study interesting. If the paper lacks a theoretical contribution, reject it (no matter how interesting the empirical contribution may be)!  This is a major obstacle, and I don’t think the esteemed editor could offer a strong counter-argument to the objection.  Editors, after all, are somewhat constrained by the reviews they get.  I think what we need is a new way to think about what makes a study valuable. We need new language to talk about research quality.

Let me put it on record that I agree with the editor who wants to see more papers published in organizational research that are problem-focused in nature.   That doesn’t, however, have to mean that they are theoretically empty. I’m also on the side of those scholars who think that what makes an empirical problem particularly interesting is that it says something about a more generalizable set of issues and ideas that span contexts and generations. As a field, this is what makes organizational theory worthwhile.  We are sensitized to a particular set of problems that link classic sociological theory to contemporary thoughts about how organizations and organizing shape our social world. It’s a pretty exciting and broad set of ideas. There are many theoretical ideas to tackle in this space.

I think the core impediment to doing meaningful research is how we have come to define a “theoretical contribution.” For many people this means linking your study to a major theory and then suggesting why some core concept is overlooked or how nobody has ever studied X or Y or Z before with this theory. Presumably, the assumption is that because no one has ever studied X, Y, or Z, studying it now will truly change the way we think about that big theory (although it never really does). Making theoretical contributions in this way has led to the proliferation of concepts that offer little value to our understanding of the world.  All we have done is to expand the number of concepts that we can use to label stuff that we see.

I think a better way to approach research is to let the empirical problems guide us. Find something to study that is truly puzzling, and try to figure out what is going on. If you find an empirical problem that is difficult to explain given your current theoretical perspective, there is a good chance that in figuring out an answer to this problem you’ll also come up with a novel theoretical insight. I would dare to suggest that the major theoretical breakthroughs of our field have been made by people who were truly puzzled by a real world problem they encountered and in trying to solve that problem they developed an original theoretical insight. For example, Philip Selznick didn’t start off by trying to find an empirical setting where he could show that co-optation led to a transformation of an organization’s values. Rather, he looked at the TVA and tried to figure out why this organization that had such great promise for realizing the ideals of a liberal political agenda was thwarted from accomplishing this mission. The answer it turns out led to his theoretical insights about institutionalization and co-optation.  John Meyer and Brian Rowan were interested in explaining why schools are constantly adopting new innovations and structural changes without ever really altering what goes on in the classroom. The result was a theory of about the myth of rationalization.  I would suggest that most of the time this is the pattern of how great theoretical insights are developed. People are puzzled by something they see in their empirical observations and by digging deeper into the data, making comparisons and doing analysis, they come up with a new insight.

Notice that I’ve changed the language here. Rather than talk about “theoretical contribution” (which has become a wasted term in our vernacular), I think it’s time to talk about “theoretical insights.”  Contribution has come to mean that you are developing a new concept, adding to an established stream of literature, etc. while insight can mean many things. It might mean that you are making a contribution to an existing stream but it could also mean that you are simply reimagining the way a particular mechanism works or questioning the existence of an empirical phenomenon that has come to be taken for granted. A valuable insight might be derived from observing that there is unexpected variance or deviation in an empirical phenomenon, which then might lead us to rethink our assumptions about that phenomenon.  Insights are plentiful in good research. We should start rewarding them.

Written by brayden king

September 12, 2012 at 3:50 pm

7 Responses

Subscribe to comments with RSS.

  1. The Academy of Management recently announced a new journal (Academy of Management Discoveries) that I think is (well, partly) trying to address these issues – http://aom.org/amd/

    Like

    teppo

    September 12, 2012 at 4:00 pm

  2. Reminds me of the old joke about the guy who died and went to heaven and was being shown around by Saint Peter. The man was struck with awe and glory for the mansions of many rooms. “Shhhh,” said the gatekeeper. “We have to tiptoe past here.” Why? “That room has Presbyterians and they think they are the only ones here.”

    The relationship between theory and experiment has been explored for many years in many fields. Perhaps Thomas Kuhn explained it best in The Structure of Scientific Revolutions. Kuhn’s mentor Paul Feyerabend was unrelenting: any so-called “empirical” investigation is really informed by theory, so all theories and all facts are imperialistic and irrelevant (or something like that).

    The reality of the situation is that a researcher undertakes an “interesting” problem because some theoretical assumptions suggested that this phenomenon is perhaps outside the normal expectation. Thus, theories suggest experiments which cause “crises” and new paradigms.

    That still leaves unanswered the question of what an “experiment” is. It seems that social sciences are more rigorous in teaching this than are the physical sciences. A recommendation from OrgTheory here led me to find this work:

    Since the Renaissance, the term experiment has been used in diverse ways to describe a variety of procedures such as a trial, a diagnosis, or a dissection … To examine changes in the textbook construction of experimental method, introductory texts in psychology, sociology, biology, and physics were surveyed during three time periods: 1930-1939, 1950-59, and 1970-79. […] … the percentage of texts with discussions of research methods increased from 50%-90% in psychology, from 25%-70% in sociology, from 20%-45% in biology, and from 16%-30% in physics. Even in the 1970s, such discussions were absent from the majority of biology and physics texts.
    “What Counts as an Experiment?: A Transdisciplinary Analysis of Textbooks, 1930-1970,” Andrew S. Winston and Daniel J. Blais. The American Journal of Psychology, Vol. 109, No. 4 (Winter, 1996), pp. 599-616.

    In my undergraduate research methods class we were taught that you must operationalize the independent and dependent variables, isolate the factors seeking out (and eliminating) artifacts and intervening variables. You must have a hypothesis which is further tested and mathematically validated within acceptable ranges and tolerances. In short, it is difficult to see how any so-called “empirical” or “ethnographic” study cannot be motivated by some theoretical framework and cannot itself indicate at the very least some refinement to accepted theory.

    Like

    mikemarotta

    September 12, 2012 at 10:08 pm

  3. The problem with this dichotomy seems for me to be that social science theories are so broad and vague that it is entirely possible to write up solution to any new problem as an application of old theory. Thus any publishable paper solving a new problem is also a rhetorical exercise in constructing new theory.

    I do not want to imply this is somehow wrong.

    Like

    henri

    September 13, 2012 at 6:10 am

  4. Thanks for this. I think this is an issue in sociology more generally, at least at many of the elite journals. I often find myself frustrated by reviewer comments that seem to ignore or play down the interesting empirical aspects of my own research as they demand a bigger theoretical contribution. I actually think this has a great deal to do with the subject matter — problem focused articles on some subjects (urban or family come to mind) are acceptable, but others are not. In any case, I very much like this idea of theoretical insights. I get a little tired of every article/book proposing new “concepts,” most of which seem like minor revisions of an older concept.

    Like

    bedhaya

    September 13, 2012 at 1:03 pm

  5. Great post, Brayden. I have to say, the editor you allude to sounds like a person of great boldness and vision. I am sure everyone would enjoy reading his/her journal, and citing it a lot.

    There are a couple of types of papers that occasionally make a big impact: (1) those that name something with an evocative term (e.g., “absorptive holes” or “structural safety” or whatever) that others can then cite for their own purposes, and (2) those that reveal something surprising or insightful about the world. Interestingly, I can’t think of many papers offhand that are widely cited because they did a great job of filling a theoretical gap, or had especially awesome regressions. (I could be wrong–anyone who wants to generate an inductive theory about this can gather a sample of impactful papers here: http://asq.sagepub.com/cgi/collection/asq_award_for_scholarly_contribution_winners )

    I wonder if there is a disjuncture between what reviewers say and what really drives their evaluations. I used to worry that reviewers would be hostile to manuscripts that were not firmly grounded in existing theory, but I have found that reviewers are quite open to papers that are just plain insightful. After all, reviewers are not some anonymous tribe that lives in an underground bunker — they are us. We all want to read insightful papers, so when they come along, we tend to be sympathetic. Perhaps when reviewers say that a paper makes no contribution to theory, they really mean “This paper is not very insightful and I did not learn anything.”

    Note that we authors are not always the best judges of how insightful their own papers are. And even the most acerbic reviewer will avoid telling an author that their work is boring. Maybe “contribution to theory” is a polite dodge.

    Like

    jerrydavisumich

    September 14, 2012 at 2:27 am

  6. Very interesting discussion here. I fully agree with the notion that a strong dichotomy between theory and empirics is not a useful way to look at the issue; and a new language like “theoretical insight” would really help. Empirical analysis has to start somewhere, and even the most trivial observation is a reflection of how our brain “theorizes” via the process of cognitive representation.

    However, I think there is a different explanation for why reviewers insist on demanding theoretical contribution – maybe it is not as much a problem in sociology or org theory, but certainly in strategy, where phenomena-driven research is arguably more accepted, there are a large number of articles that are obviously the result of “I have this amazing dataset, I haven’t a clue what theory or insights I have in mind when I got it, but let me pretend I collected the data because I was puzzled by something, theory or phenomena, and let me also pretend that the data I collected just happen to address that puzzle so damn well.” Call me cynical, but I think in top strategy journals, the bulk of the articles read like this kind of pointless research efforts – obviously much more motivated by scoring the A hits than conducting actual scholarship. The outcome is we have tons of papers in top strategy journals that no practitioners would find remotely interesting (or outright ridiculously out of touch with reality), and yet it contributes little or no advancement on the theory side – that is, it bores the bejesus out of both scholars and practitioners.

    Reviewers push back on this type of papers and say they lack “theoretical contribution,” and just like what Jerry said above, that might be a polite way of saying “who cares” to the author.

    Like

    Peter MacCormack

    September 14, 2012 at 3:57 am

  7. This post illustrates the concept of phronetic social science (Flyvbjerg). An important point you make is about our science talk. The language that we use to describe our work has become imbued with meanings that tacitly affect scholarship. Perhaps normal science in the social sciences has become more about generating a language than uniting theory and practice. Reimagining science talk may be a good first step toward deconstructing the dichotomies that exist in social science, generating insight, and moving towards praxis.

    Like

    Mindy Duncan

    September 18, 2012 at 1:45 pm


Comments are closed.

%d bloggers like this: