Nightmare after nightmare: students trying to replicate work

Trying to replicate a published paper can be challenging. Often, authors do not remember their variable codings, models specifications, or where their files are. My students of the Cambridge Replication Workshop just finished their final assignments – here are their ‘horror’ stories.

Rejected from journal because it’s a replication?

This is a case from psycholinguistics, but it’s interesting for the political science replication debate as well. The journal Applied Psycholinguistics rejected a re-analysis of a paper after a 20-month review process. The original paper was published in the same journal.

The fear of being scooped: share your work

Following up on my post on how to publish as a grad student, here’s a video on scooping anxiety. If you share your work, people will know it’s your idea. You will be cited, not scooped.

Replication correspondence: “Let me see what I can dig up”

Nicole,

(…) I will definitely send a .dta file when I can clean it up a bit. I am less sure I can track down the .do files, but let me see what I can dig up. (…)

Best,
___________

After Reinhart-Rogoff: A new model for replication policy from the natural sciences

It took three years to replicate the economic paper “Growth in a Time of Debt” by Carmen Reinhart and Kenneth Rogoff. In these three years, the paper and its errors were cited widely in the field and heavily relied on by politicians. Why has no one found out earlier? Because the data were not online. Such delay in cross-checking work can be prevented. By using the natural sciences as a model, journals in political science and economics can adopt a better replication policy. The key point: journals should require authors to provide their data set when submitting a manuscript.

What Reinhart-Rogoff means for the replication debate

Economist Paul Krugman commented in the New York Times on the Reinhart and Rogoff replication scandal. He wrote: only after R-R “allowed” others to see their own data set, the spreadsheet errors could be detected. I don’t think it should be a matter of “allowing” anyone to see replication data of published work. Here are three four key points on what we can learn from this for the replication debate.

Replication scandal: We might not need austerity measures after all

Economists Carmen Reinhart and Kenneth Rogoff claim in “Growth in a Time of Debt” that countries with debt-to-GDP ratios above 90 percent have a slightly negative average growth rate. Their paper is one of the most cited by governments to justify austerity measures – and it might be wrong, as a replication paper shows.

Publishing as a grad student: replication helps

A new article in Political Science & Politics claims that replication can be a “potential route” to get a class paper published early in your career. Below are tips from the paper how to make this work.

Replication chains: Political tolerance taken apart

How would you like it if someone replicated your paper? Will it help you, and give you more citations, or will it cause rage? Some authors defend their original work against replication by writing an ‘answer’ paper, and they might claim that the replication was fundamentally flawed or contained statistical and reporting errors. Here’s another interesting replication chain.

7 good reasons why we should replicate existing work

Why should we replicate published papers? Isn’t it a waste of time, given that journals and universities expect you to create original research? In the Cambridge Replication Workshop I discussed this with my students. Here are 7 good reasons why we should replicate existing work – beyond the ‘getting published’ argument.