what i did and did not say about critical thinking

Two weeks ago, we had quite a discussion about critical thinking. There was one strand of debate the baffled me. Thomas wrote the following:

“If I don’t teach critical thinking, then what do I teach?” you ask. “Turns out that there is a simple answer: sociology.” It sounds good at first. But what you’re really saying is that you won’t acknowledge a kind of critique that isn’t sociological. That’s certainly the line Goffman’s defenders are taking. If you think she’s wrong, they say, it’s because you don’t understand how ethnographers think.

That is not what I was saying. What I said is this:

Aside from a very simple general rules of thumb, such as “don’t be emotional in arguing” or “show my your evidence,” the best way to be improve your thinking is to learn from those who have spent a lifetime actually trying to figure out specific problems.

I did not say: “only trust experts.” I said: “learn from.” In other words, there are people who have encountered certain problems and they have tried and tested the ideas that might have occurred to you. In the processes of trying out those ideas, they have probably learned important things about the phenomena that you are looking at. You should probably learn those lessons. Once you have absorbed those ideas, you are free to criticize as you will. No one has a monopoly on truth, even the experts. But that doesn’t mean experts are clueless fools. Certainly, “critical thinking” must have a place for learning from other people.

50+ chapters of grad skool advice goodness: Grad Skool Rulz ($2!!!!)/From Black Power/Party in the Street

Written by fabiorojas

September 17, 2015 at 12:01 am

Posted in education, fabio

12 Responses

Subscribe to comments with RSS.

  1. Fabio:

    Interesting discussion, with excellent points by both you and Thomas. This is yet another example of why I prefer blogs to twitter: instead of battling one-liners, we have an engagement, one paragraph at a time.

    Regarding the question of how (or whether) to teach critical thinking:

    As a statistician, I teach critical thinking by trying to embed it within statistics: that is, I teach specific methods using tools such as prior distributions, predictive checks, and graphs which embody critical thinking by juxtaposing data, assumptions, and conclusions in ways that we hope will reveal problems with existing inferences.

    One thing I’ve been thinking more about is to spend more time in class explaining why certain statistical tools that are associated with critical thinking, don’t really do the job they’re intended to do. I’m thinking here of hypothesis tests, which nominally are tools for falsification but in practice typically are used as empty exercises in confirmation.

    I wonder if you could do something similar in a sociology class: embed the acts of criticism into a sociological framework so that instead of criticism being an outside, extra-sociological activity, it is actually presented as part of the sociological endeavor. You can then discuss issues such as whether Alice Goffman should’ve engaged in more self-criticism, and the role of pre- and post-publication criticism in the reception of her work.


    Andrew Gelman

    September 18, 2015 at 5:39 am

  2. I don’t think we need to argue about what we did or did not say because I really do think we disagree about the issue. When you say ” Once you have absorbed those ideas, you are free to criticize as you will,” you are saying, (I take it) “if you have not taken on board the ideas of people who have spent a lifetime actually trying to figure out specific problems then your general objections about logical inconsistencies, basic scholarly craftsmanship (like proper citation), or conflicts of interest are not relevant to evaluating a particular piece of research”. And your argument for this is that there just isn’t any discipline-independent (i.e,. non-“specific”) kind of critique. While I disagree with it, I think I understand this view, and I don’t take it to imply that we should “trust the experts”; rather, I take it to imply that we should not bother with criticism that comes from even very intelligent non-experts, nor should we teach our students how to address such criticisms.

    Also, it seems to be way of rejecting informed lay criticism by the people who are being studied, as we saw in the comments of your post on Deep Springs.

    My view is that “critical thinking” is matter of holding even very specialized research accountable to the basics tenets of rationality and, to an extent, civil discourse. Being capable of (general) critical thinking is part of being open (general) rational critique. It’s not a skill that I think we should just assume comes with the sociological toolbox … except of course in the sense that I think non-specific “critical thinking” should be in the tool box, and should be taught to sociology students.


    Thomas Basboll

    September 18, 2015 at 11:32 am

  3. Fabio and Thomas:

    Regarding the disputed phrase, “Once you have absorbed those ideas, you are free to criticize as you will,” let me just say that I think a possible solution is to undo the “mind-body” or “church-state” separation of research methods (or “sociology,” or “important things about the phenomena that you are looking at”) and criticism. My proposal is to place criticism within the scientific, or social-scientific, enterprise, rather than thinking about it as something coming from outside, or as something that is tacked on at the end.

    To put it another way:

    1. We can engage in criticism as social scientists, using statistical (or, more generally, research) methods.

    2. Knowing that we will be criticizing our own work, and that others will criticize it, will and should affect how we conduct our research. This is related to the saying, “To understand the past, you must first know the future,” or the statistical principle that the design of a study is best considered in light of how the data will be analyzed.

    A psychology researcher once wrote to me a mini-manifesto, disparaging my efforts as a critic. He wrote:

    There are two kinds of people in science: bumblers and pointers. Bumblers are the people who get up every morning and make mistakes, trying to find truth but mainly tripping over their own feet, occasionally getting it right but typically getting it wrong. Pointers are the people who stand on the sidelines, point at them, and say “You bumbled, you bumbled.” These are our only choices in life. If one is going to choose to do the easier of these two jobs, then one should at least know what one is talking about. Sorry, but I think by and large you don’t.

    I think this dualism is counterproductive. If, instead of dividing the world into “bumblers” and “pointers” (and let me not even comment on the ridiculousness of a research psychologist making such a categorization and saying “these are our only choices in life”), we were to consider “bumbling” and “pointing” to be two essential activities conducted by any scientist–and, for that matter, if we were to recognize that one can and should spend lots of time criticizing one’s own work (that is, “pointing” at our “bumbling”) and that we can and should consider criticism to itself be an ongoing development (that is, “bumbling” in our “pointing”)–then, I think our research and our criticism could improve.

    To draw a humble analogy from my experiences on the street: If each of us were to spend some time as a pedestrian, some time as a bicyclist, some time as a bus rider, and some time as a car driver, than I think we’d all be able to interact more efficiently and considerately. But if we associate each person (or, in the sociology example, each area of expertise) with only one role, we get all kinds of trouble, as indicated by various psychology and biology researchers who can’t seem to handle open criticism.


    Andrew Gelman

    September 18, 2015 at 2:08 pm

  4. Except for being uncomfortable with the image of “placing criticism within the scientific, or social-scientific, enterprise” rather than, say, situating sociology within the broader setting of critical thinking, I agree with Andrew. I’m assuming the sociologist drives a particular kind of vehicle and has to learn that there are other kinds vehicles (different disciplines) too, from bikes to to trucks, and also different kinds of traffic like that of pedestrians.

    Some of this “traffic” in ideas is not “sociological” but still something the sociologist has to respect as properly critical. And the rules of that criticism must also be understood by the sociologist. Again, it’s not enough for the sociologist to offer the pedestrian a lift, or even just to stay off the sidewalk. There are times when the needs of drivers simply have to yield to the needs of others, and drivers have to be taught what those needs are so they can constrain their behavior.

    I’ll site the Deep Springs case as an example again here, because I think it shows how Fabio’s intolerance for non-specialized, “pedestrian” criticism works in practice.


    Thomas Basboll

    September 18, 2015 at 3:03 pm

  5. “Once you have absorbed those ideas, you are free to criticize as you will,” you are saying, (I take it) “if you have not taken on board the ideas of people who have spent a lifetime actually trying to figure out specific problems then your general objections about logical inconsistencies, basic scholarly craftsmanship (like proper citation), or conflicts of interest are not relevant to evaluating a particular piece of research”.” Wow, the confirmation bias is strong in this one. At least it is explicit. But still ironic.


    Carsten B

    September 18, 2015 at 6:26 pm

  6. Thomas: I clicked through and read the Deep Springs post and the comments.

    Fabio: What’s up with that? The Deep Springs people flat-out disputed some of the things you claimed. But maybe they’re not telling the truth. I have no idea. Here we are, 3 years later: what are your current thoughts on the matter?

    Liked by 1 person

    Andrew Gelman

    September 18, 2015 at 7:13 pm

  7. Thomas: You write of “situating sociology within the broader setting of critical thinking” rather than “placing criticism within the scientific, or social-scientific, enterprise” (as I put it).

    I guess I’d say that each contains the other. Setting aside social science for a moment, let me talk about statistics, which is my core area of expertise. I think that (a) statistics can be set within the broader setting of critical thinking, and also (b) criticism can be placed within the statistical enterprise. Indeed, I’ve personally written a lot about both of these and also used both (a) and (b) in my applied work.

    In a sense, I think we’re being impeded by the implicit and naive use of set theory as model of ideas. It’s very natural to say that if we have two ideas, X and Y (in this case, X could be critical thinking and Y could be sociology) and to think that X can be separate from Y, or X and Y can overlap, or X can be a subset of Y, or Y could be a subset of X. But in many ways I think it is appropriate to think of X as a subset of Y, and Y as a subset of X. This sounds weird, but only because we’ve so internalized the idea of sets and Venn diagrams.

    To put it another way, 1 + 1 = 1 sounds weird too, but not if you’re talking raindrops.


    Andrew Gelman

    September 18, 2015 at 7:18 pm

  8. It’s true I’m thinking about this like a Venn diagram. Your raindrops get me thinking of the Taoist taijitu, Andrew. Critical thinking is the black “eye” in the white “fish” of sociology. And sociology is the white “eye” of the black “fish” of critical thinking. I’m not sure I agree, but it’s an interesting notion.


    Thomas Basbøll

    September 18, 2015 at 9:00 pm

  9. Andrew and Thomas B: Wow, bringing up Deep Springs (which is weird, as you will see in the Spring). I am not sure what Thomas meant meant by “pedestrian” criticism. The issue in that post was:

    – I claimed that Deep Springs finally succumbed to the social trend of co-ed education that started in the 1960s after years of resistance.

    – The commenter disputed some details, but did not dispute the basic fact that Deep Springs admitted women very late in the game. They did dispute the framing I provided.

    – Then, in my response, I provided evidence showing that yes, there was indeed contention about the issue. The evidence was from journalists who covered DS in earlier decades. I also admitted that I got some details wrong but that the detail were not germane to the main point (e.g., arguing about the cohort size at DS doesn’t mean I was wrong about the resistance to female DS students).

    Later, in a different post, they disputed my characterization of DS as subject to the institutional pressures of higher ed.

    So, if by pedestrian, you mean “quibbling over details that aren’t relevant” to the argument, then yes, I am intolerant of them. Guilty as charged. But, the DS commenters actually went and read a little about institutional isomorphism and then responded. That is not pedestrian and is commendable, even if I think their response is incorrect.

    Take that as you will.

    Liked by 1 person


    September 18, 2015 at 9:10 pm

  10. Fabio, I think it would be commendable of sociologists if they directly addressed the objections of the people whose practices they explain, rather than requiring that those objectors go and read a little sociology first.

    I know that by putting it that way I’m turning your commendation for doing something into a requirement to so, but I don’t think this completely misses the spirit of what you’re saying. In my view, there is a connection between not teaching critical thinking except as sociology and deflecting non-sociological criticism with some suggested reading (in this case about institutional isomorphism).

    In your comment, it seems like you want to establish criteria of “relevance” for particular kinds of argument. And you’re pretty consistent about saying that an argument is relevant if (and, it seems, only if) it is grounded in some specific sociological competence, not a more general ability to reason.

    I believe that this sort of thinking has led to the widespread use of doubtful methods in the social sciences that lead to absurd results that are, technically, i.e., in line with “the specific forms of critique” that are currently being practiced in sociology, “significant”. One of things that Andrew has gotten me to see much more clearly is that we cannot replace all actual reasoning (about, say, effect sizes in the populations we model) with specific methods of analysis. Sure, we can sometimes formalize an aspect of critical thinking and import it into our methodologies. But these innovations and patches come from somewhere, and we should teach students what that is.


    Thomas Basbøll

    September 19, 2015 at 12:08 pm

  11. Fabio:

    This is a minor sidebar but just let me explain that I had never heard of Deep Springs; I was just curious about it after reading that old blog discussion.


    Andrew Gelman

    September 21, 2015 at 2:10 am

  12. Hi Fabio, after reading your description of your disagreement with Deep Springs, I went back and reread their comments looking for an indication that they “went and read a little about institutional isomorphism and then responded”, and that the argument ended with a polite disagreement over the role of isomorphism in bringing about the change. The closest I could find was their reference to the “the pleasing fantasy that no matter what the facts, when they talk theory, college professors never have to own up to silly mistakes in their blogs”. (Maybe you could link to that “different post”?)

    That is, it seems to me that you are still characterizing what they themselves see (and I also see) as a substantive critical objection as, instead, “quibbling over details that aren’t relevant to the argument”. Your comment leaves the impression that they’ve come around a little, and are now willing to grant that their original factual issues aren’t relevant. I don’t see that anywhere in the exchange, but I guess things will become clearer this spring? (Like Andrew, I’m curious to hear where things stand today.)


    Thomas Basboll

    September 21, 2015 at 11:21 am

Comments are closed.

%d bloggers like this: