Category Archives: Replication & Teaching

“People like stories” A short film about reproducibility

janz replication spiegelhalterWe need mathematical help to tell the difference between a real discovery and the illusion of one. Fellow of the Royal Society and future President of the Royal Statistical Society, Sir David Spiegelhalter visits Dr Nicole Janz  to discuss reproducibility in scientific publications. Here’s the film:

Continue reading


replication and extension projects: making class more interesting and useful

A Princeton course instructor assigned replications to his students. Here’s his recap and advice to other teachers:

Wheels on the bus

This semester I taught the second course in my department’s quantitative methods sequence that is required for all of our graduate students: Advanced Data Analysis for the Social Science. Sociology departments around the country all have a pretty similar required sequence. In teaching the course this time, I tried to modernize it so that it would train students for the future (not just the present or the past).

One big aspect of this modernization was requiring students to complete a project where they replicate and extend an already published paper. Overall, this change was a big success, and I’d recommend that other classes also try it. In this post, I’ll share some of what worked about the project and how I will do it better next time. I’ve also made all of the materials that we’ve used available on the class website.

View original post 951 more words

Navigating the Winds of Change: A Grad Student’s Journey Through the Replication Crisis, by Matthew J. Samson

One of the graduate students in my Replication Workshop writes about replication, the reproducibility crisis and validity in experimental psychology.
Continue reading

Re-Blogged: Solid science – How graduate students foster research transparency

replication nicole janz in mindRe-Blogged from the InMind blog (and written by me): Reproducibility is seen as the gold standard for solid science. However, three are few incentive to work transparently, and even less incentives to conduct replication studies. To change this, more and more teachers are assigning replication studies to graduate students as a class assignment. Will this turn early career researchers into witch hunters?
Continue reading

Tagged ,

Checklist for a Gold Standard Replication

Screen Shot 2014-12-14 at 15.37.50Many social scientists agree that replication studies are necessary to provide quality standards in research. But how does a good replication study look like? Here is the checklist I will use in my Replication Workshop.
Continue reading

Tagged , , , ,

Teaching reproducibility for stats beginners – give me seven ideas!

ReplicationI’m teaching reproducibility as part of a lecture for statistics beginners in the social sciences. I reserved seven slides for that and I have 15 minutes. What would you include?

I’m thinking of these topics, and I would really need input on how to rank these, or additions on what I missed. Please comment below or on twitter (@polscireplicate).

  • Definition of reproducibility and research transparency
  • Recent scandals in the social sciences
  • How to save your project files in a transparent way (structuring your files)
  • Software tools 1: Rstudio and Rscripts
  • Software tool 2: knitr and R markdown
  • What can happen if you don’t work reproducibly

If you know of lecture slides or other materials, please share the links so I can give them to my students. I will also publish a link list on this blog. Thank you!

Tagged , , ,

Share your replication: Political Science Replication Initiative

Screen Shot 2014-08-29 at 5.56.57 PMEarlier this year, we sent out a survey on replication in political science to the Political Methodology (PolMeth) mailing list. Our survey results, which we wrote up in a post on the Monkey Cage blog, indicated quite a bit of interest in a site to share replication studies.
Continue reading

Tagged , , ,

What is a failed replication?

failed replicationA lot of original authors are concerned about their reputation when their work is replicated, and the replication fails. But when can we actually label a replication as “failed”? And how should we deal with unhappy original authors who feel ‘bullied’? Continue reading

Guest Post: Publishing a replication? Definitely worth repeating, by Chris Hartgerink

Chris Hartgerink is a research master student at Tilburg University, the Netherlands. I invited him to write a guest post about his experience of publishing a replication project. It turns out that, among many other aspects of doing a replication study, one of the main take away points was that a replicator must be reproducible as well. He explains here why it is important to always have a second assessor on all the analysis code before submitting to a journal.
Continue reading

Tagged , , ,

Reproducible research on coursera: Week 2 introduces knitr and R Markdown

I’m doing the free Coursera course on reproducibility by Johns Hopkins University to improve my own teaching. Week 2 introduces knitr and R Markdown, two core tools to create reproducible research.

Continue reading

Tagged , , , , , , ,