Yale psychology lab retracts papers after failed self-replication

The Retraction Watch blog just published a story of a “scientist doing the right thing at significant professional cost”: Yale’s Laurie Santos could not replicate her own papers, and decided to request a retraction by the journals.

The Yale researcher, who works on evolutionary origins of the human mind by comparing the cognitive abilities of human and non-human primates, had published two papers (one in Developmental Science and another in the Journal of Personality and Social Psychology) based on observations of monkey behavior.

Commenting on the Retraction Watch blog, Laurie Santos explains that her lab found the errors due to her lab’s policy “regularly try to replicate our own effects with new samples at our field site, just to be sure the effects we report hold in different samples.”

The research is based on videos of behavior of monkeys, which were coded for data analysis. The lab tried to replicate the results and planned to extend the samples. This process went all the way back to the initial coding of the videos: “[J]ust to be on the safe side, we decided to go back to the initial videos (…) and have new coders recode them all from scratch.”

The self-replication failed

Once the coding errors were detected, the lab retracted both papers based on the same data (and codings), and informed the university, which investigated the case. The university “found that [the coder] was not guilty of any misconduct or negligence. It seems it was just human error”, says Santos.

Santos describes the whole experience on the Retraction Watch blog:

“Having to retract papers is a scientist’s worst nightmare. Especially in the current climate in psychology right now (e.g., Hausergate, Stapelgate, etc…), this is pretty much the most awful thing that could happen to a PI. But I also hope that this awful situation can – at least in some sense – serve as a positive example of correcting the scientific record. We would have never caught the coding error without replicating the initial JPSP effects (which particularly given the subject access that plagues primate cognition work is something that more and more scientists need to do). And as soon as we found the problems, we immediately went back and checked all the other datasets too. I’m obviously embarrassed that we didn’t catch all this earlier, but I’m still glad that we caught it when we did.”

Discussion in the community

As the comments below the article at Retraction Watch show, not everyone agrees that Santos reacted to issues with the data in a ‘model’ way and should be ‘praised’. Here are some points raised:

  • One commenter says that the methods section in the papers says there was a second coder who agreed with the first one. Therefore, the story of a “single bad rater” is not quite right.
  • The transparent reaction of Santos might just be a very good PR strategy to prevent further damage by “spinning an ‘error’ story.”
  • Santos says it’s the worst thing that can happen to a PI. However, it might also be an example of “just another PI throwing a poorly trained/supervised student under the bus.”

Highest scientific standards

A study recently found that scientists reward authors who report their own errors. Citation frequencies decline less drastically when authors self-retract their paper – compared to retractions that were not self-reported. By making the replication problems public, initiating an investigation, requesting a retraction and commenting on the whole process openly, Santos and her lab definitely did the right thing – even if we do not know if really just a single coder made errors or what the whole story is.

The retracted papers

  • Neha Mahajan, Jennifer L. Barnes, Marissa Blanco, Laurie R. Santos “Enumeration of objects and substances in non-human primates: experiments with brown lemurs (Eulemurfulvus)” Developmental Science, Volume 12, Issue 6, pages 920–928, 2009
  • Mahajan, N., Martinez, M., Gutierrez, N. L., Diesendruck, G., Banaji, M., & Santos, L. R. (2011). The evolution of intergroup bias: Perceptions and attitudes in rhesus macaques. Journal of Personality and Social Psychology, 100, 387– 405. doi: 10.1037/a0022459

Read the original blog

… on Retraction Watch.

Advertisements
Tagged , , , , , ,

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: