orgtheory.net

sorry, peter klein, social science is a fuzzy business

Evil twin conspirators Peter Klein and Nicolai Foss have been ragging on management scholars for using ill-defined concepts, like routines and leadership. When I read the post, I thought it was odd – aren’t all social science concepts a simplification of complex behaviors? What is democracy? What is a market? What is an organization? These are all broad, inherently fuzzy concepts, but it doesn’t mean it’s dumb to talk about them.

I was reminded of a valuable exercise that Don Levine once did in a graduate seminar. Thoroughly versed in the history of social thought, he could take nearly any concept offered by students and then give a long discussion of how many very smart people had argued over its definition, formalization, and operationalization. I even offered my own example: among mathematicians, it’s recognized that there are multiple and incompatible definitions of things like sets and numbers. True, it may not matter to a typical mathematician, but there’s a fundamental ambiguity about basic math that allows one to logically construct some pretty exotic version of the integers.

Back to social science, I’d argue that the underlying concepts of any social science are inherently vague. Formalizing them allows you to ignore the ill-defined nature of the process so you can make a clean academic model. It doesn’t address the underlying mess. Here’s an example to twink an economist like Peter Klein: price. If you define price as a real number (or function) then it’s pretty clear. Heck, economists even have the “law of one price,” which says if things are efficient, identical products cost the same.

But if you start with “what you have to give to get X,” then things get hairy pretty fast:

  • in a barter system, there is not a single number, but a list of exchange rates with different commodities.
  • in many tribal cultures, there are significant gift economies, where there is not an obvious, or stable, “price” or exchange ratio, just some generalized expectation of reciprocal obligation
  • Many markets have haggling, the “price” depends not on information about supply and demand or other economic information, but on personal ability to argue. In other words, sticker price, or a single price, is sort of meaningless, at best you have a range of prices depending on how hard people push various personal interactions.
  • Some markets are so differentiated that producers offers prices based on minute personal characteristics that leads to a vague price. Think car dealers – advertised price, dealer sticker price, haggled price, price conditional on subtle variations in the car, and price contingent on local markets conditions can be substantially different. In this case, price is a Jackson Pollock splatter, not a well defined number.
  • there seems to be price dispersion in some markets, and this is the subject of research by many economists

We can define away the ambiguity of the situation by saying “there is a single price – if there is no friction, everyone has identical information, if we stick to intra-temporal comparisons, and equivalent abilities of buyers and sellers to bargain in identical ways.” Well, of course, but you’ve just defined aways alot of interesting things about prices! So, Peter amd Nicolai, I’ll stick with my fuzzy management concepts.

Written by fabiorojas

May 13, 2009 at 3:27 am

39 Responses

Subscribe to comments with RSS.

  1. Right on the nose. Every field of experts adopts its own domain of symbolic rules and procedures. The minimum number of experts needed to constitute a “field” and the minimum number of symbolic rules and procedures needed to constitute a domain are the variables we can use to adjust the granularity in the fuzziness to a domain.

    But the disagreement between economists and sociologists may come down to only one rule. Economists generally won’t trust a rule unless it applies to something that can be counted.

    Like

    Michael F. Martin

    May 13, 2009 at 4:11 am

  2. Routine, the construct that ignited Nicolai’s comment, may be hard to measure (e.g., how routinized is something?) but it’s not that hard to observe. Go into any organization and sit around and watch stuff happen and sooner or later you’ll observe a routine. Probably a lot of them. Procedures of all sorts (e.g., this is how you change the toner in the copy machine) are routines. Sometimes they’re even written down in very precise language. So, while difficult to measure, a routine is about as real as any other organizational phenomena.

    The problem I have with routines is that they probably don’t explain as much variance as some OT scholars would have them explain. It’s fairly difficult to pinpoint the effect of routines on performance differences, for example, and yet in the strategy literature routines are given quite a bit of causal significance. I imagine this is why Nicolai has issues with routines.

    Like

    brayden

    May 13, 2009 at 4:31 am

  3. True, but shouldn’t Nicolai’s post be then called “routines aren’t a good explanation of performance” rather than “One More Ill-Defined, Un-Measured (?) Core Construct: Routines?” If you read the post, it’s really about his dislike of routines since they are fuzzy:

    “The boundaries of the concept are, even for management research, highly ill-defined and virtually everything in an organization, save for physícal capital, that has some degree of stability has been called a routine by some author”

    Like

    fabiorojas

    May 13, 2009 at 4:36 am

  4. This is probably the strangest post I have read on orgthery.net
    1)I wrote the post that inspired it. Yet, it is directed to Peter.
    2) I have no difficulties accepting that core social science concepts are quite fuzzy at the edges. I guess the same holds for Peter. Actually, we are both sympathetic to Austrian economics, and a number of Austrians have made similar arguments.
    3) Still, there are limits to how imprecise we should be. At any rate, we shouldn’t celebrate imprecision as a virtue.
    4) We should strive to make core concepts so precise that they are applicable in empirical research. It is hard to do any quant research on routines because it not clearly defined, not dimensionalized etc. Brayden is right in saying that of course we can “observe” routines, but that is partly because so much has been put under the routine umbrella.
    5) The price example in the post does not establish that “price” is ill-defined. It suggests — quite correctly — that there is a lot of price dispersion in real markets.

    Like

    Nicolai Foss

    May 13, 2009 at 4:40 am

  5. Hi, Nicolai – a few responses…

    1. Trust me, this is by no means the strangest post on this blog. Just ask Teppo about our strange obsession with exotic pets (or life companions, as I call them).

    2. My bad – it should’ve been called “sorry, nicolai foss,…”

    3. I think we have a real disagreement – I was not claiming that certain concepts are “fuzzy at the edges,” I am claiming that they are often fuzzy on a basic level.

    4. In my book, price dispersion = fuzzy price = ill defined. I think price in economics is analogous to temperature in physics: there is no such as thing as “temperature.” What you see on a thermometer is really a statistical summary of a complex system. In practice (like in a doctor’s office), it’s ok to treat it like a single number, but any decent physicist will tell you it’s an average of many things. And any engineer will also tell you that the measured temp is good enough.

    Or let me phrase my view: “price of the taco I bought this evening” – well defined (one transaction), “price of tacos” – terribly complex.

    5. Hold on, I take it back – now that I’m writing it, the price of even a single transaction can be vague. Think about the credit card purchase – the ultimate price of an item is decomposed into the sale price (fixed), interest (variable), plus credit card charges (variable). In other words, for credit based purchases I can only estimate the amount of money I will pay to claim the item.

    Like

    fabiorojas

    May 13, 2009 at 4:58 am

  6. Fabio, I guess I’m confused too. Not about the title of your post — you want to attract readers, so it makes sense to include celebrity names like “Angelina Jolie” or “Peter Klein” in the post title. But about your example of price. Price is very precisely defined in all variants of economics I’m familiar with. A price is an exchange ratio between two goods. In a barter system, the price of a loaf of bread might be one flask of beer. In a monetary economy, goods and services exchange against money, so the price is denominated in the monetary unit. Your discussion seems to confuse the definition of price and the causal explanation of particular prices. Whether the law of one price does or doesn’t hold, whether the price of a particular bundle of services (the good itself, a loan on the funds used to pay for it) can be decomposed into separate prices of the individual components, whether we can aggregate prices of heterogeneous goods and services into a “price level,” and so on are interesting issues, but have nothing to do with the definition of price itself. Likewise, I would argue that the core concepts of economics — scarcity, opportunity cost, preferences, marginal benefits and costs, the laws of supply and demand, etc. — can all be defined quite precisely. They are not at all like routines, capabilities, leadership, etc.

    Like

    Peter Klein

    May 13, 2009 at 5:20 am

  7. Great discussion. Very quickly re routine: Brayden’s spot-on that routines can be observed in organizations, and they can also be mapped and measured. While routines can be highly specific to particular organizations and types of organizations, there is invariably overlap that creates an opportunity for qual and quant analysis. Practitioners actually do this a good bit.

    Like

    josephlogan

    May 13, 2009 at 7:43 am

  8. I think Nicolai’s basic point is being missed. The primary issue shouldn’t be whether or not the concept of “routine” is vague. The question should be: has it ever been measured in an organization? I think it was Popper who pointed out that science deals with the necessary vagueness of all concepts by stipulating procedures for measuring the quantities they are supposed to refer to.

    I really like Nicolai’s suggestion to replace “ill-defined” (as a term of abuse) simply with “un-measured”. The norm would be: don’t introduce a concept without a measuring instrument to go with it. (There’s a history of physics to write around that norm, I think. It’s probably already been done.)

    So I would challenge Brayden’s point. Yes, there’s an everyday sense in which you can “observe” routines and convince yourself organizations are full of them. But what do they do in organizations? Are there too many in this one? Not enough in this other one? Well, you’d have to *count* them.

    If you’re not going to say something about how increases and decreases in routine actvities affect organizations, what’s the point of having the concept? But if you’re not goint to measure routine, how are you going to say such things?

    Like

    Thomas Basbøll

    May 13, 2009 at 12:34 pm

  9. […] I hold to the basic Misesian position that quantitative empirical analysis is a complement to, not a substitute for, other forms of knowledge acquisition such as a priori theorizing and Vershtehen. Needless to say, this doesn’t mean I approve of fuzzy constructs in social-science research. […]

    Like

  10. What is exactly meant by measure? How can we measure without a clear definition?

    Is it necessarily to be quantified? If so, why things can’t be quantified can’t be meaningful?

    Like

    passerby

    May 13, 2009 at 2:57 pm

  11. Passerby, I think your first two questions are the essence of Professor Foss’ original point. Ill-defined constructs are doubly difficult to measure. We have enough difficulty measuring those that we have agreed definitions for (I think of affect and identity, for example).

    Strictly speaking, measurement is necessary to empirical analysis. Unmeasured things CAN be meaningful — like “faith”. They just don’t lead to coherence in management research.

    Like

    REW

    May 13, 2009 at 3:44 pm

  12. Not the easiest reading, but I think Jean-Luc Marion’s written the best philosophical critique on this question of fuzziness. Marion famously argued with Derrida about the possibility of a gift (though Marion’s best response is in Being Given not in that famous exchange). The issue can be perhaps best thought in terms of commodity fetishism as applied to ontology and/or epistemology, on my view.

    Also, though I haven’t read it all the way through, I think John Law’s After Method: Mess in Social Science Research is also a very insightful (and very accessible) discussion of these issues.

    Like

    Robert Couch

    May 13, 2009 at 5:23 pm

  13. One underlying issue here may be that one cannot make a measurement of a time-dependent phenomenon arbitrarily precise in both time and frequency domains. In other words, if a routine is a phenomenon that recurs in time, then the more and more precise we get with measuring its average frequency, the less and less precise we can be about exactly when a routine will occur.

    The problems with making measurements of value — i.e., the underlying constituents of price — are further complicated by the asymmetries in the history and forecast of individual preferences and current allocation of resources. (I.e., in addition to frequency, one has to take into account bounded rationality and asset-specifity in determining price.)

    Like

    Michael F. Martin

    May 13, 2009 at 7:15 pm

  14. @Thomas, re: history of physics.

    A lot of this work has been done. The interesting part is that instead of introducing a concept with a measuring instrument to go along with it, most of the time measuring instruments were solutions looking for a problem or a conceptual underpinning (especially around the 17th-18th c). Thermometers and barometers are an especially good case of this. You could tell a similar story with the history of statistics (cf. Daston or Hacking).

    Of course, nowadays this is reversed in physics, but there is a division of labor in matching instruments with concepts.

    Like

    dr

    May 13, 2009 at 7:26 pm

  15. Yes, a lot of interesting theorizing can be done around already reliable measurements, bringing them together to predict/infer other (indepedently measurable) values.

    Ever since I spent some time looking at the history of the lever (i.e., Archimedes) I have thought that physics is really a kind of hermeutics of machines–an interpretation of things in functional arrangements. Physics ultimately explains how the laboratory equipment works.

    With the lever you have a way of exerting a measurable force at one end on a measurable mass at the other with a pivot at a measurable distance from each.

    Like

    Thomas Basbøll

    May 13, 2009 at 9:45 pm

  16. Fabio, do we mark this day, 13 May, as the day you jumped the shark? “Concepts” are only as “vague” and “messy” as the mind that authors them. If minds are more vague and messy in the social sciences, it says more about minds in the social sciences than it does about concepts. Or so goes my 2 bits.

    Like

    a different passerby

    May 13, 2009 at 9:47 pm

  17. Thomas,

    All measurements involve counting of something.

    Not all phenomena that exist can be counted.

    Therefore there are some phenomena that exist that cannot be measured.

    Whether a person who concern himself with everything that exists, whether it is measurable or not, can still call himself a scientist, I will live for others to say.

    Like

    Michael F. Martin

    May 13, 2009 at 9:56 pm

  18. [troll] I’m just glad that economics leads the way for the rest of us with its exemplary application of precisely defined, non-circular conceptual tools such as aggregate production functions. [/troll]

    Like

    Kieran

    May 13, 2009 at 11:45 pm

  19. Diff: Jumped the shark? I’ve been hang gliding over the sharks for months…

    Like

    fabiorojas

    May 13, 2009 at 11:46 pm

  20. Hi Michael, yes, but what do we then mean by “concern oneself with”?

    I think there’s a legitimate argument to be made for sticking to the measurable *in science*. Is “organization studies” the name of a science? Perhaps this is the question Nicolai is raising. You may want to leave it to others to answer it, but that doesn’t make it a bad one.

    I recently read Lance Sandelands’s “Argument for God From Organization Studies” (JMI, 12 (2): 168-177), for example. “To refuse God because He cannot be measured by the objective standards of rational science is rank scientism,” he says. “If we are lovers of truth, we must be open to truth however it is conveyed” (176).

    I don’t think that’s right at all. We can refuse to include God in our theories of organization without denying His existence. Indeed, a religious argument for such a refusal can be made. After all, are we really going to be so presumptuous as to *theorize* God’s influence on organization?

    The point can be generalized to all unmeasurable-but-real things. It is out of respect for the unmeasurable that we should not theorize it. Measurement keeps our theories properly humble.

    Like

    Thomas Basbøll

    May 14, 2009 at 7:25 am

  21. Deities aside, are you making an argument that we should not try to theorize the (currently) unmeasurable out of humility?

    Like

    josephlogan

    May 14, 2009 at 7:28 am

  22. Well, if no one had brought up God (was that me?) I don’t think I would put it like that.

    But we should not proceed to theorize without at least imagining a measuring instrument. There are, of course, “theoretical entities” in physics that are still waiting for their respective (respectful?) measuring instrument. We definitely need to theorize in advance of (or beyond) our measurements. Nicolai is pointing out that there are some concepts that have been with us for decades though their objects have never been measured.

    Physicists, I should note, are quite good at distinguishing theoretical predictions from experimental results. They don’t make claims or propose technologies on the basis of realities not yet measured. In organization “science” we lack that humility. We make all sorts of claims about the importance of “routines” (for good and ill). I was actually surprised to learn (from this post) that no measures exist to support those claims.

    Like

    Thomas Basbøll

    May 14, 2009 at 7:52 am

  23. Does “measurable” means being able to count/qualify them? Or simply, observable and verifiable?

    Like

    passerby

    May 14, 2009 at 1:38 pm

  24. sorry quantify, not qualify.

    Like

    passerby

    May 14, 2009 at 1:39 pm

  25. Thomas,

    You made a subtle but imprtant shift in addressing my question. My question was about people and how they label themselves and others. Your answer seems more directed to an abstract body of information (“science”). The two have some relationship, but are obviously not the same.

    Like

    Michael F. Martin

    May 14, 2009 at 2:13 pm

  26. Sorry about that, Michael. I didn’t realize you were asking a question. I took both posts (Nicolai’s and Fabio’s) to bring some interesting methodological and conceptual concerns together (such concerns do belong together). I hadn’t seen the people and labels angle (identity politics?). What was your question?

    Passerby: I think “measurable” means being able to assign a value to an observation, normally one that can then be compared with values assigned to other observations. I suppose simple existential statements like “there are routines” could verified as such by observation. But I think being able to count observable entitites is a kind of minimal condition. If you are able to say “there are routines in this organization” you should be able to say how many routines there are. This will normally be impossible unless you say *what kind* of routine you are talking about. That is, the definition needs to be more precise. Otherwise “there are routines in this organization” becomes a tautology (all organizations have routines by definition).

    Like

    Thomas Basbøll

    May 14, 2009 at 4:08 pm

  27. Well I didn’t ask directly, but you answered so well that I had to claim I’d asked. ;-)

    One person can simultaneously respect and follow the rules and procedures of two different symbolic domains of knowledge. I’d call it cognitive dissonance if the two domains are logically inconsistent. But if the two are not inconsistent, such as when one domain applies only to things countable and another to both things countable and uncountable, then the same person might label themselves two ways without affecting the logical consistency of the domain of knowledge to which they affiliate themselves.

    Like

    Michael F. Martin

    May 14, 2009 at 5:50 pm

  28. Are we lumbering toward the view that the study of organizations is both an art and a science? Drat, these labels…

    Like

    josephlogan

    May 14, 2009 at 6:51 pm

  29. If you’re implying that precision is a virtue of science and not of art, I would hope that is not where we’re going. I think Nicolai is right to suggest that one way of improving precision is by insisting on measurement. I’m not sure what the alternative proposal is. Like Nicolai, I hope we don’t simply make a virtue of imprecision.

    Like

    Thomas Basbøll

    May 14, 2009 at 7:15 pm

  30. Precision in art and in science are much to be desired. I have to confess that I am losing the plot on where this rich conversation is heading. For what it’s worth, I think that routine is measured, and that art and science are a part of org theory. Just because things are fuzzy doesn’t mean they aren’t worth trying to understand them.

    Like

    josephlogan

    May 14, 2009 at 7:31 pm

  31. At the end of his post, Nicolai asks: “do you know of any attempts to grapple empirically with routines in the sense of actual measurement? Are there any scales out there?” We might regain the plot with some examples of measurements and scales.

    Like

    Thomas Basbøll

    May 14, 2009 at 8:15 pm

  32. I do want to point out that some economics concepts are precisely defined yet unobservable, such as preferences. They can usefully be treated as latent constructs that are manifest in some observables (e.g., preferences are inferred from observed behavior). The latent construct can be useful for theorizing, yet not directly useful for empirical work.

    Kieran, “precisely defined” is not the same as “sensible.”

    Like

    Peter Klein

    May 15, 2009 at 3:49 am

  33. Oh, and Fabio, this is your Best. Thread. Ever. (Best-titled thread, anyway.)

    Like

    Peter Klein

    May 15, 2009 at 3:49 am

  34. Really cool hypothesis based on this post: blog posts submitted in the form “Sorry, X, Y is a Z business” are likely to generate generate healthy debate. Not only would that be a routine, but it would also be measurable.

    Like

    josephlogan

    May 15, 2009 at 9:51 am

  35. Define “healthy”…

    Like

    REW

    May 15, 2009 at 5:40 pm

  36. […] the idea of routines. Teppo’s collaborators at orgtheory.net responded to this by saying that all social science concepts are fuzzy, to some […]

    Like

  37. […] a comment » I think I’m broadly on Fabio’s side when it comes to the question of the vagueness of concepts in the social sciences. I think my main caveat is that, based on the […]

    Like

  38. […] paper addresses one of my pet peeves, the expansive use of “capital” to describe any ill-defined substance that accumulates and has value. Hence knowledge, experience, and skills become […]

    Like


Comments are closed.