Checklist for a Gold Standard Replication

Many social scientists agree that replication studies are necessary to provide quality standards in research. But how does a good replication study look like? Here is the checklist I will use in my Replication Workshop.

Teaching reproducibility for stats beginners – give me seven ideas!

I’m teaching reproducibility as part of a lecture for statistics beginners in the social sciences. I reserved seven slides for that and I have 15 minutes. What would you include?

I’m thinking of these topics, and I would really need input on how to rank these, or additions on what I missed. Please comment below or on twitter (@polscireplicate).

• Definition of reproducibility and research transparency
• Recent scandals in the social sciences
• How to save your project files in a transparent way (structuring your files)
• Software tools 1: Rstudio and Rscripts
• Software tool 2: knitr and R markdown
• What can happen if you don’t work reproducibly
• …

If you know of lecture slides or other materials, please share the links so I can give them to my students. I will also publish a link list on this blog. Thank you!

Replication Workshop: What frustrated students the most, and why they still liked the course

The Cambridge Replication Workshop 2013/14 just finished. In eight sessions, graduate students replicated a published paper and learned about reproducibility standards. This is a summary of student feedback on data transparency and the course itself. Some were extremely frustrated, a few dropped out, and those who stayed found the course “fantastic” and “incredible”.

Reblog: Social science and replication

My colleague Alex Sutherland (Twitter: @as2140) and I have written a short piece on replication for the Alliance for Useful Evidence Blog.

Getting published by replicating work

There are three motivations to replicate someone else’s work. First, you learn statistics and methods. Second, you provide a service to the community by advancing knowledge and validating results. Third, and most importantly for many, if you add value, you can get it published. Here are some success stories from the social sciences about papers that got published by replicating existing work.

Setting up a replication workshop – suggestions needed

I’m starting a replication workshop with my statistics students. During my lectures in basic statistics I found that students were most engaged in lab sessions when we worked with data sets. I want to build on that and use replication as a new teaching tool, and I will report about this on my blog. My first challenge: How to pick a paper that’s simple enough for statistics beginners?

#overlyhonestmethods Best tweets about tricks and manipulation in social sciences

This week thousands of researchers used the hashtag #OverlyHonestMethods to admit how they or their colleagues manipulated data, gave up on experiments, included nonsense because a reviewer requested it. These confessions are hilarious, and a bit unsettling. Here’s a selection of tweets interesting for political and social scientists. After reading those, we don’t have to wonder anymore why no one provides their replication data. The main themes right now for social and political scientists seem to be: didn’t read citations; pleasing reviewers; statistics manipulation; please don’t replicate my study; qualitative and mixed methods issues; my supervisor told me so; forging significance; I don’t know what I’m doing.