7 good reasons why we should replicate existing work

Why should we replicate published papers? Isn’t it a waste of time, given that journals and universities expect you to create original research? In the Cambridge Replication Workshop I discussed this with my students. Here are 7 good reasons why we should replicate existing work – beyond the ‘getting published’ argument.

Understand analysis

“As virtually every good methodology text explains, the only way to understand and evaluate an empirical analysis filly is to know the exact process by which the data were generated and the analysis produced.” (Gary King, Replication, Replication 1995)

 

Learn how to find solutions

“(…) seeing replication not as an end in itself but as a means for acquainting yourself with the methods used in a study, the original author’s line of thinking, the complications he or she must have faced, and the solutions they devised to those problems.” (Gary King, interviewed by Price 2011, To Replicate or Not To Replicate?)

“[R]eplicating cutting-edge results (…) provides object lessons about the scientific process.” (Frank and Saxe 2012, Teaching Replication)

 

Verify results

“I think it’s important if you’re building on results in a field to replicate them, to be able to not just verify them but to understand how these results actually came about,” she says. “That makes you a little more educated in terms of how you’ll be able to build on those results. It’s easier to extend the science if you know how it got to the place where it is today.” (Victoria Stodden, Columbia University, interviewed by Price 2011, To Replicate or Not To Replicate?)

 

See through shaky foundations of academic knowledge

“The remarkable difficulties students have in replicating published articles teaches more about the state of the literature, and conveys more about the sometimes shaky foundations of academic knowledge, than reading all the published literature one person could possibly consume on his or her own.” (Victoria Stodden, Columbia University, on her syllabus for STAT 8325: Topics in Advanced Statistics: Fall 2012)

 

Hunting for errors can be fun

“I did [replication] as a student in Colorado, where we replicated “Inequality and Insurgency” by Muller and Seligson in the American Political Science Review. There was a typo in one of the tables and the challenge for the students was to find the typo. That was a great exercise.” (Todd Landman, University of Essex, Political Science Replication Blog)

“[R]eplicating cutting-edge results is exciting and fun; it gives students the opportunity to make real scientific contributions (provided supervision is appropriate)” (Frank and Saxe 2012, Teaching Replication)

 

Jump to the research frontier

“Replication is the easiest way to jump to the research frontier. In replicating an established argument, students learn by doing—they employ sophisticated models, get hands-on experience with datasets, and learn of the shortcuts and miss-representations that are often taken to secure publication results.” (Jonathon Moses, Norwegian University of Science and Technology, Political Science Replication Blog)

 

Get published

You can “write a publishable article by beginning with the replication of a previously published article” (Gary King, How to Write a Publishable Paper as a Class Project). There is now a range of replication articles that made it into high ranking journals in political science in the last years.