Category Archives: replication frustration

Nightmare after nightmare: students trying to replicate work

Screen Shot 2013-05-14 at 4.09.38 PMTrying to replicate a published paper can be challenging. Often, authors do not remember their variable codings, models specifications, or where their files are. My students of the Cambridge Replication Workshop just finished their final assignments – here are their ‘horror’ stories.
Continue reading

Advertisements
Tagged , , , , ,

What Reinhart-Rogoff means for the replication debate

Screen Shot 2013-04-19 at 9.38.04 AM
Economist Paul Krugman commented in the New York Times on the Reinhart and Rogoff replication scandal. He wrote: only after R-R “allowed” others to see their own data set, the spreadsheet errors could be detected. I don’t think it should be a matter of “allowing” anyone to see replication data of published work. Here are three four key points on what we can learn from this for the replication debate.
Continue reading

Tagged , ,

Replication scandal: We might not need austerity measures after all

Screen Shot 2013-04-17 at 8.08.34 AMEconomists Carmen Reinhart and Kenneth Rogoff claim in “Growth in a Time of Debt” that countries with debt-to-GDP ratios above 90 percent have a slightly negative average growth rate. Their paper is one of the most cited by governments to justify austerity measures – and it might be wrong, as a replication paper shows.
Continue reading

Tagged , , , , , ,

Only 18 of 120 political science journals have a replication policy

Screen Shot 2013-03-08 at 6.37.24 PMA recent article in “European Political Science” found that out of 120 political science journals only 18 have replication policies. The authors found that the existence of a data policy was correlated with the impact factor, type of audience, and age of the journal. Their analysis confirms that the state of reproducibility and political science is substandard.
Continue reading

Tagged , , , , , , , ,

“My doctoral institution bought the data”

Nicole,

(…) The ____ dataset I am afraid I can’t give you, because my doctoral institution bought it and it is proprietary. (…)

Best,
___________

 

More replication correspondence

I collect more responses from original authors here.

Tagged , , ,

“Can’t make data available because of contractual agreement”

Dear Nicole,

(…) We can’t make our ____ data available because of our contractual agreement with ___. (…)

Best wishes,

________

 

More replication correspondence

I collect more responses from original authors here.

Tagged , , , , ,

Replication Frustration in Political Science

replication_head_sm“I don’t have a ready-made dataset.” – “We don’t have the R code for our paper available.” – “I’m travelling. I will definitely send the replication data when I can clean it up a bit.” These are just some of the answers I received when asking authors for their replication data in political science. Only few sent me replication-ready data and almost no one sent me their code or .do file. This is a serious case of replication frustration!
Continue reading

Tagged , , ,