Replication and transparency in political science – have we made any progress?

When a range of top political science journals signed a statement to enforce transparency in 2014 (JETS statement), there was an immediate backlash by qualitative researchers. Hundreds of scholars signed a petition against strict transparency rules asking for clarification. Then the LaCour scandal happened, where a political scientist fabricated a study and pretended to withhold his data because of confidentially. Another wake-up call. Where is the debate in political science now? 

Advertisement

Replication in international relations: New research and blog

Nils Petter Gledisch and I just published a guest blog post about replication in international relations at the OUP blog. The blog is based on new research in the field, which we published as a symposium in International Studies Perspectives. We negotiated with OUP that all seven articles will be free access for a few weeks. Make sure to download all the pdfs before they go behind paywall again.

Coding errors can be avoided

An article in the American Journal of Political Science was corrected after the coding of a political attitude variable was accidentally the wrong way around. Pre-publication cross-checks by the authors and the journal, as well as publication of the original data and variable transformations can avoid such problems.

Getting the idea of transparency all wrong

Following an article in the New England Journal of Medicine, which portrayed scientists who re-use data as parasites, we now hear more on this from Nature. Apparently, data transparency is a menace to the public. The Nature comment “Don’t let transparency damage science” claims that the research community must protect authors from harassment by replicators. The piece further infects the discussion about openness with more absurd ideas that don’t reflect reality, and it leads the discussion backwards, not forward. 

Don’t review articles that withhold data

Journal editors can enforce replication policies. Authors can decide to work transparently. Most initiatives for open science and reproducibility agree that editors and authors are  are the key actors to enforce the gold standard of research integrity. However, peer-reviewers can use their leverage as well: just say you will only review an article once the author provides the data.

Reproducible Research in Biomedical Science – We’re not there yet

A new PLoS Biol aper on reproducible research practices across the biomedical literature examines if authors provide all data, code and funding information. The results are devastating.

How Reporting Statistical Significance Causes p-Value Hacking

A new article by researchers at the University of Amsterdam shows that publication bias towards statistically significant results may cause p-value misreporting. The team examined hundreds of published articles and found that authors had reported p-values < .05 when they were in fact larger. They conclude that publication bias may incentivize researchers to misreport results.

Guest Post: The Replication Road – Scientific Detour or Destination? By Nate Breznau

Guest Post: I asked Nate Breznau, an empirical sociologist, to write about his experience replicating published work – the challenges, benefits, and how he got published.

Political science should not stop young researchers from replicating

In a blog post by Jesse Singal at the New York Magazine, one of the students who tried to replicate LaCour‘s findings on gay marriage said that young researchers are often actively discouraged from replicating work.

Journal editor: It is up to the author to release their data

I received this email from a journal editor in political science after I asked about their replication policy. My original email is below.