diederik stapel
This case against social psychologist Diederik Stapel is something else. Unbelievable. Here’s the Science summary: Dutch ‘Lord of the Data Forged Dozens of Studies (including updates).
One of the updates is an English version of the formal case – including a response by Stapel at the end (pdf). He promises to provide a longer response by Monday.
Unbelievable, indeed. Here in the Netherlands, the social science community is shocked; people hardly talk about anything else. Not only is the impact disastrous for those directly involved (imagine finding out that half of your dissertation is based on fictitious data), but the impact is also incredibly wide; even though I am not a social psychologist, I know already two people directly affected by Stapel’s actions. That is not to mention the reputational damage to (social) science in general, in the public opinion.
Nevertheless, the whole affair poses some interesting questions from an orgtheory point of view. How can it happen that a professor gets away with this for years, right under the eyes of his close colleagues? The report has some interesting pointers on this, including the fact that he was often the sole supervisor of grad students and junior researchers.
Second, why were his fabrications (which in hindsight appear to have been rather clumsy) never discovered in the peer review process of very reputable journals (including Science!)?
Rense
November 3, 2011 at 8:48 am
Stapel worked with sociologists as well – for example Siegwart Lindenberg.
Doctoral students and co-authors have got to be just devastated by this.
teppo
November 3, 2011 at 1:35 pm
I have one hypothesis about the second question raised by Rense. Peer reviewers will hound an author about theoretical development, conceptual model representations, missing citations and references, and statistical techniques. Data are nearly always accepted as gods’ own truth, save for flaccid examination of first and second moments or correlation coefficients. Reviewers rarely comment on data structure (binary, ratio, ordinal, …) and even more rarely ask for the data in order to examine them. Then, with the active collaboration of reviewers and editors, one may apply ever-increasingly arcane models and estimation techniques in order to make something pop out at the end that will support the hypotheses. One recalls the phrase, “Pay no attention to the man behind the curtain” from the Wizard of Oz. That is the norm for social science with respect to data. How could it NOT happen that unexamined data (and their putative collection) would be fabricated?
Randy
November 3, 2011 at 3:15 pm
So we finally have our own Schon scandal? Now we can be called a science.
Guillermo
November 3, 2011 at 4:28 pm
From a trust perspective, Stapel’s first response seems to tick the first box in the trust repair process. I look forward to his longer response on Monday.
Angelos Alexopoulos (@angelos_alex)
November 3, 2011 at 8:56 pm
Randy, I’m afraid you’re right. Most of us (reviewers) are not trained to detect suspicious results. Perhaps the best solution is to make availability of data (even if at some point in the future) a requirement for publication. At least that would have prevented some of Stapel’s boldest moves, namely not even fabricating a raw dataset but just reporting fabricated results. Of course it would still be possible to fabricate entire datasets, but this is much harder to do and there are techniques to detect fabricated data.
By the way, NWO (the Dutch NSF) recently announced that open access to data will be a default requirement for all funding from now on. Not a coincidence, probably….
Rense
November 4, 2011 at 2:47 pm
Ugh. I’ve already been involved in one review where I was asked to review “the data.” Believe me, the job of reviewer is already time-consuming enough. The last thing this profession needs is a requirement that senior people in the field spend their time checking other people’s data sets and computer programs. The only logical way to institute this type of checking is to assign graduate students to do it, for whom the exercise would be an actually valuable learning experience. I have long thought that our field gives too little attention to the value of replication and that the correct model for this is as exercises for grad students. It is my understanding that this is the model for lab sciences: when a “big” finding is published, the next step is for lots of labs to see if they can replicate. I see this as most usefully done when something is published or accepted for publication. Perhaps the journals could create a registration process where graduate programs/students say which things they are in the process of replicating. This of course would be linked to a requirement that publications entail providing access to the underlying data. There are some kinks to be worked out regarding restricted data sets, but I still think this is the only logical path. And I am CERTAIN that asking journal reviewers to re-run people’s analyses is NOT the logical path.
olderwoman
November 4, 2011 at 4:47 pm
Just to be clear: I wasn’t suggesting that reviewers re-running analyses is the logical path. My point was that requiring access to raw data would make cheating a lot less attractive, because there is at least the possibility to look more closely at suspicious results. That doesn’t mean it should be done every time.
That being said, I completely agree that we need more replication, need reward attempts at replication, and need ask for replication of a spectacular result before we believe it. That would to a large extent remove the incentive for fraud, because you won’t get famous after all if your results can’t be replicated.
Rense
November 4, 2011 at 9:39 pm
Apparently Diederik Stapel has relinquished his PhD – http://news.sciencemag.org/scienceinsider/2011/11/update-disgraced-dutch-pyschologist.html
teppo
November 11, 2011 at 8:41 pm