response to gelman on what retraction does and does not do

In our recent discussion about retraction, Andrew Gelman wrote the following:

I’m on record as saying that retraction is not much of a solution to anything given that it’s performed so rarely.

So I agree with you, I guess, and I’d probably go further and say that we can’t realistically expect papers that are fraudulent or fatally erroneous. Again, the problem is that there are so many papers that are fraudulent or fatally erroneous, that most of them aren’t gonna get retracted anyway.

We have to get away from the whole idea that, just cos a paper is published in a serious journal (even a top journal), that it’s correct or even reasonable. Top journals regularly publish crap. They publish good stuff too, but they also publish a lot of crap. And, to the extent that retraction is a way to “protect the brand,” I’m against it.

This comment made me think about the problem with litigation – while it may help the plaintiff achieve an outcome, it rarely solves any broader problem. This is because taking people to court is a lengthy, expensive and inefficient process. Retraction is really similar. It is simply not a tool meant for more systematic monitoring of academic work. It is a blunt tool meant only for really extreme cases.

What would I suggest? 1. Encourage openness and replication. 2. Institute rules so people can share data. 3. Create systems were discussions of papers can be appended to papers. These are all less expensive and more decentralized ways to monitor work.

50+ chapters of grad skool advice goodness: Grad Skool Rulz ($4.44 – cheap!!!!)/Theory for the Working Sociologist (discount code: ROJAS – 30% off!!)/From Black Power/Party in the Street / Read Contexts Magazine– It’s Awesome 

Written by fabiorojas

October 4, 2017 at 4:01 am

2 Responses

Subscribe to comments with RSS.

  1. Fabio:

    I agree with your take and your suggestions. Indeed, one could say that retraction is not just like a legal proceedings, it’s almost like the death penalty, in that journals seem to have a strongly asymmetric attitude in which keeping an erroneous paper is considered to be mildly regrettable, while retracting a paper that retains some value is considered a terrible mistake. Retraction also seems to be considered a sort of punishment of the authors. And, like the death penalty process, retraction procedures can be costly and time consuming.

    Without making any judgment on whether all this is a good thing, I think we can accept that, at least for existing journals, retraction is no more a solution to incompetent and even fraudulent research than the death penalty is a solution to street crime.

    Liked by 2 people

    Andrew Gelman

    October 4, 2017 at 12:17 pm

  2. I’m thinking a lot about retraction because I recently had to post a retraction to a working paper after it was discovered that an error had been made very early in the process of constructing the data. I completely agree that a culture of replication is essential. I also think that we need to develop a culture that acknowledges human imperfection is normal and that researchers acting in good faith may publish errors and create a non-punitive mechanism for allowing errors to be reported and corrected. This same culture of recognizing the risk of error as normal would promote both full documentation of procedures (so everyone can look for errors) and make us all more careful about double-checking our own procedures.

    Liked by 2 people


    October 4, 2017 at 5:47 pm

Comments are closed.

%d bloggers like this: