Ethics, Scandals and Self-Correction: My Predictions for Reproducibility in 2018

What’s going to happen in the reproducibility world in 2018? What are the new trends to push transparency, especially in the social sciences? And most importantly, what’s the next big challenge to be tackled? Here are my top predictions, collected with the help of reproducibility folks on twitter.

Advertisement

Guest Post: The Replication Road – Scientific Detour or Destination? By Nate Breznau

Guest Post: I asked Nate Breznau, an empirical sociologist, to write about his experience replicating published work – the challenges, benefits, and how he got published.

Journal editor: It is up to the author to release their data

I received this email from a journal editor in political science after I asked about their replication policy. My original email is below.

Open peer review: What are the benefits and pitfalls?

Should journals publish review reports alongside a paper? At the meeting “The future of scholarly scientific communication” at the Royal Society it seemed that there is a general agreement: yes – we need more transparency. However, opinions were divided if reviewers’ names should be published as well.

Many academic journals hesitate to publish replications

While most political scientists would agree that replications and cross-checks advance knowledge, many academic journals hesitate to publish replications. John Ishiyama, University of North Texas, discusses how replication studies can get published to create more incentives to do replications.

Would journals publish only failed replications?

Scholars in political science commonly agree that we need more researchers doing replications of published work. But how to motivate them? In the recent symposium on the topic in PS: Political Science & Politics (Vol 47, Issue 1), Thomas M. Carsey discusses concrete steps researchers and journals can take. This blog post is part of a series of posts about the PS symposium, which summarizes and discusses the current state of reproducibility in political science.

After Reinhart-Rogoff: A new model for replication policy from the natural sciences

It took three years to replicate the economic paper “Growth in a Time of Debt” by Carmen Reinhart and Kenneth Rogoff. In these three years, the paper and its errors were cited widely in the field and heavily relied on by politicians. Why has no one found out earlier? Because the data were not online. Such delay in cross-checking work can be prevented. By using the natural sciences as a model, journals in political science and economics can adopt a better replication policy. The key point: journals should require authors to provide their data set when submitting a manuscript.

Publishing as a grad student: replication helps

A new article in Political Science & Politics claims that replication can be a “potential route” to get a class paper published early in your career. Below are tips from the paper how to make this work.

Only 18 of 120 political science journals have a replication policy

A recent article in “European Political Science” found that out of 120 political science journals only 18 have replication policies. The authors found that the existence of a data policy was correlated with the impact factor, type of audience, and age of the journal. Their analysis confirms that the state of reproducibility and political science is substandard.