What is a failed replication?

failed replicationA lot of original authors are concerned about their reputation when their work is replicated, and the replication fails. But when can we actually label a replication as “failed”? And how should we deal with unhappy original authors who feel ‘bullied’? Continue reading

Against publication bias: Politics journal invites pre-registered papers

After fields like Psychology have explored pre-registration of studies, now Political Science follows the trend for more transparency in research. A high ranking journal, Comparative Political Studies, plans a special issue in which all papers have to be pre-registered and are submitted without the results section. I asked Michael Findley, one of the editors behind the initative, about publication bias and transparency in the field. Continue reading

Tagged , , , , , ,

Guest Post: Publishing a replication? Definitely worth repeating, by Chris Hartgerink

Chris Hartgerink is a research master student at Tilburg University, the Netherlands. I invited him to write a guest post about his experience of publishing a replication project. It turns out that, among many other aspects of doing a replication study, one of the main take away points was that a replicator must be reproducible as well. He explains here why it is important to always have a second assessor on all the analysis code before submitting to a journal.
Continue reading

Tagged , , ,

Guest Post: Two simple things to make your research more reproducible, by Thomas Wallis

In this guest post, experimental psychologist Thomas Wallis (University of Tübingen) proposes two simple ideas how you can make your work more reproducible.
Continue reading

Tagged , , ,

Guest Post: Be a better scientist – how to make your scientific output more credible, by Thomas Leeper

Thomas Leeper, a political scientist at Aarhus University, recently wrote about where to store your replication data. In his second post, he explains what kind of data to archive, and why that makes you a better scientist. His post is packed with concrete steps and state-of-the-art software tips.
Continue reading

Tagged , , , , ,

Fast and detailed answer from original author

A while ago, I tried to replicate tables from a textbook on panel data analysis to learn more about fixed effects, random effects and hybrid models. While code and files for the book were available on a webpage for the book, I still had some questions. I received a detailed reply only three days after I wrote to one of the authors.

Continue reading

Tagged , ,

Methods vs Insights #4: The four stages of a project (and the fifth you should avoid)


Reproducibility is one of the hardest stages in a research project. Here are tips for writing successful papers in any field, re-blogged from the Scientific B-Sides Blog.

Originally posted on Scientific B-sides:

Methods vs Insights is back. Today with a discussion of general research practice.

Most projects in my lab take years from start to finish. So it is important for me to manage the expectations my students and postdocs may have. Here is a plot I have developed to discuss the different stages of a scientific project with them and to prepare them for what’s ahead.

The four stages of a scientific project: Explore! Dig! Refine! Sell! And the stage you want to avoid: Waste! Plus the prevalent emotion in each stage and the key skill you will need to successfully navigate it.

The four stages of a scientific project: Explore! Dig! Refine! Sell! And the stage you want to avoid: Waste! Plus the prevalent emotion in each stage and the key skill you will need to successfully navigate it. (x-axis it time, y-axis is work you’ve put in.)

View original 1,352 more words

Tagged , ,

“Replication Bullying:” Who replicates the replicators?

A recent special issue in Social Psychology adds fuel to the debate on data transparency and faulty research. Following an innovative approach, the journal published failed and successful replications instead of typical research papers. A Cambridge scholar, whose paper could not be replicated, now feels treated unfairly by the “data detectives.” She says that the replicators had aimed to “declare the verdict” that they failed to reproduce her results. Her response raises important questions for replications, reproducibility and research transparency.

Continue reading

Tagged , , ,

Guest Post: Archive your data or they die with you, by Thomas Leeper

Thomas Leeper is a political scientist at Aarhus University and focuses on public opinion, political psychology, and experimental methods. I invited him to write a guest post because he is a contributing developer for the rOpenGov and rOpenSci open source software projects. He also wrote the very useful dvn package that connects the statistical software R and the data archive Dataverse. In his post, Leeper explains why there is no reproducibility without proper data archiving.
Continue reading

Tagged , , , ,

Reproducible research on coursera: Week 2 introduces knitr and R Markdown

I’m doing the free Coursera course on reproducibility by Johns Hopkins University to improve my own teaching. Week 2 introduces knitr and R Markdown, two core tools to create reproducible research.

Continue reading

Tagged , , , , , , ,

Get every new post delivered to your Inbox.

Join 1,794 other followers