Nils Petter Gledisch and I just published a guest blog post about replication in international relations at the OUP blog. The blog is based on new research in the field, which we published as a symposium in International Studies Perspectives. We negotiated with OUP that all seven articles will be free access for a few weeks. Make sure to download all the pdfs before they go behind paywall again.
The integrity of science is threatened in many ways – by direct censorship; by commercial, political, or military secrecy; by various forms of publication bias; by exorbitant journal subscription fees that effectively deny access to the general public; by cheating and falsification of results; and by sloppiness in the research process or the editorial process prior to publication. There isn’t a single antidote to all these problems, but transparency goes a long way in relation to many of them.
One important way to promote transparency and quality control in published research is to require that systematic data be made available for replication studies.
In political science, the replication idea received a major impetus some twenty years ago in a widely cited and debated article by Gary King. In 2002, editors of international relations journals joined forces at a panel at the annual convention of the International Studies Association and signed a joint statement that pledged the journals to a set of minimum replication standards and urged other editors to follow their lead. The statement recommended not just the release of all data used, but also a codebook and program files that would make it possible to reproduce the published results. All files should be submitted electronically for posting on a website maintained by the journal for that purpose, but could of course also be posted elsewhere.
Although these and other journals did post substantial numbers of datasets, several studies have shown that the journals in political science and international relations have been slow in implementing satisfactory replication practices. Many leading journals lack a clear policy on replication and some that do lag behind in the implementation. In many cases, scholars, when been left to their own devices, post the data on their own websites. Such links are often unstable and are not updated when the owner moves to another institution. In some cases, data are updated and replication becomes impossible because the original dataset is not preserved. Even the four political science and international relations journals that signed the 2003 declaration have holes in their replication files.
Nevertheless, the field seems to be moving forward, even if slowly. … [read the whole article at the UOP blog].
Check out the symposium in International Studies Perspectives, open access for a few weeks, here.
The contributions are:
Replication in International Relations by Nils Petter Gleditsch; Nicole Janz
Preplication, Replication: A Proposal to Efficiently Upgrade Journal Replication Standards by Michael Colaresi
Data Policies, Data Management, and the Quality of Academic Writing by Alexia Katsanidou; Laurence Horton; Uwe Jensen
Bringing the Gold Standard into the Classroom: Replication in University Teaching by Nicole Janz
Replication and the Manufacture of Scientific Inferences: A Formal Approach by Fernando Martel García
Grounded Theory Generation: A Tool for Transparent Concept Development by Todd N. Tucker
Replication: Why, Where, and How? A Synopsis by Nils B. Weidmann
Political Science Replication 