Transparency and data access – these public goods are crucial principals for science. But why don’t researchers implement them? Political scientist Joseph K. Young discusses incentives for reproducibility, and how he is tracking down old replication data for the leading international studies journal ISQ.
Transparency, access, confidence. These three public goods are crucial principals for science. From the moment one begins doing quantitative research, these values are preached. Gary King for at least two decades has advocated building community norms to encourage access to data, transparency in our methods, and confidence in our results through replication. King has built an open data initiative, trained hundreds of Harvard students with these norms, prolifically published his work, then practiced what he preached. What about the rest of us?
Pressure to publish
Young scholars (and
older experienced scholars) are motivated to publish, to get tenure, to work quickly, and to do what is in their self-interest. Preparing replication materials, verifying the results one publishes, and then disseminating these data and replication materials, are often set to the aside in the inevitable rush to career advancement. This is the more benign version. Regardless, there are few incentives, material or otherwise for scholars to follow these best practices.
New data repository for ISQ
With this setting in mind, the lead editor of International Studies Quarterly (ISQ), Dan Nexon tasked me with dealing with this conundrum in the context of the flagship journal for international studies. ISQ, like many other journals, requires authors of quantitative articles to submit data and replication materials with their published articles. We have a new data repository, where anyone can download these materials. These data and practices span from the previous editors at Indiana to present.
Tracking down replication data for older articles
There are a few spots of missing data, but we are actively recovering and filling in those holes. If you received an email from me, congrats on the ISQ publication! I apologize in advance for the annoyance. We either don’t have or don’t know where your data are and want to make them available.
Online replications of forthcoming ISQ articles
To improve reproducibility and replication for our newly published articles, ISQ online has a symposia section where established scholars replicate a forthcoming ISQ article and suggest future avenues for research. Our initial examples include a discussion of autocratic cooperation in the international system and the reasons for US humanitarian assistance.
We have an upcoming symposium on foreign direct investment. This symposium will feature an article by Andrew Kerner. Two leaders in the field, Quan Li from Texas A & M and Nate Jensen from George Washington University, replicate his work and probe and prod the findings, exemplifying how transparency and access increase our confidence in published results and build a foundation for future quality research.
We hope these symposia will provide encouragement to replicate, serve as examples of the value of openness and transparency, and show potential avenues for new ideas and exchanges.
A slightly longer version of this guest post appeared at the ISQ discussion forum.
About Joseph Young
Joseph K. Young is Associate Professor at American University with a joint appointment in the School of Public Affairs and the School of International Service. His research seeks to understand the cross-national causes and consequences of political violence. He has published numerous peer-reviewed articles across academic disciplines, including political science, economics, criminology, and international studies. He is a contributor to the Political Violence @ a Glance blog, and an associate editor for ISQ Online. Twitter: @JosephKYoung.