Best of replication & data sharing Collection 2

These are the best pieces I came across in the last months on replication, reproducibility & data sharing. While not strictly on political science, they are inspiring and worth discussing. Collection #2 (5 March, 2013 – May 27, 2013).

How to make your work reproducible

When reproducing pubished work, I’m often annoyed that methods and models are not described in detail. Even if it’s my own project, I sometimes struggle to reconstruct everything after I took a longer break from a project. An article by post-docs Rich FitzJohn and Daniel Falster shows how to set up a project structure that makes your work reproducible.

Nightmare after nightmare: students trying to replicate work

Trying to replicate a published paper can be challenging. Often, authors do not remember their variable codings, models specifications, or where their files are. My students of the Cambridge Replication Workshop just finished their final assignments – here are their ‘horror’ stories.

Rejected from journal because it’s a replication?

This is a case from psycholinguistics, but it’s interesting for the political science replication debate as well. The journal Applied Psycholinguistics rejected a re-analysis of a paper after a 20-month review process. The original paper was published in the same journal.

The fear of being scooped: share your work

Following up on my post on how to publish as a grad student, here’s a video on scooping anxiety. If you share your work, people will know it’s your idea. You will be cited, not scooped.

Replication correspondence: “Let me see what I can dig up”

Nicole,

(…) I will definitely send a .dta file when I can clean it up a bit. I am less sure I can track down the .do files, but let me see what I can dig up. (…)

Best,
___________

After Reinhart-Rogoff: A new model for replication policy from the natural sciences

It took three years to replicate the economic paper “Growth in a Time of Debt” by Carmen Reinhart and Kenneth Rogoff. In these three years, the paper and its errors were cited widely in the field and heavily relied on by politicians. Why has no one found out earlier? Because the data were not online. Such delay in cross-checking work can be prevented. By using the natural sciences as a model, journals in political science and economics can adopt a better replication policy. The key point: journals should require authors to provide their data set when submitting a manuscript.

What Reinhart-Rogoff means for the replication debate

Economist Paul Krugman commented in the New York Times on the Reinhart and Rogoff replication scandal. He wrote: only after R-R “allowed” others to see their own data set, the spreadsheet errors could be detected. I don’t think it should be a matter of “allowing” anyone to see replication data of published work. Here are three four key points on what we can learn from this for the replication debate.

Replication scandal: We might not need austerity measures after all

Economists Carmen Reinhart and Kenneth Rogoff claim in “Growth in a Time of Debt” that countries with debt-to-GDP ratios above 90 percent have a slightly negative average growth rate. Their paper is one of the most cited by governments to justify austerity measures – and it might be wrong, as a replication paper shows.

Publishing as a grad student: replication helps

A new article in Political Science & Politics claims that replication can be a “potential route” to get a class paper published early in your career. Below are tips from the paper how to make this work.