I recently received an email where a graduate student is trying to ask original authors for data. There is some evidence that authors withhold data due to time constraints (Tenopir et al. 2011), but they may also decline to share data because they fear a damaged reputation when a replication of their work fails (Lupia and Elman 2014). From my experience in the Cambridge Replication Workshop, authors are more willing to share their data when the replicator is perceived as trying to be helpful rather than cross-checking results. Here are my tips.
Email to Nicole:
I am a doctoral student at _______ and for my dissertation I am exploring how to extrapolate ideas of reproducibility and transparency to policy analysis. (…)
As part of my dissertation I am trying to replicate __________. [The original authors’] report is quite clear and I believe that I will get pretty close to their results. However, I would really like to get more support from them (i.e. access to the code and/or more documentation).
In my first conversation with them they strike me as being pretty defensive and scared that I was after a “gotcha!” moment. My primary interest is to demonstrate how some of the practices of reproducibility and transparency can be used in policy analysis to better connect public policy and research, not to highlight potential flaws in their report. Do you have any advice/reference on how to approach this issue (i.e. don’t come across as the gotcha! person?).
In my experience, most people will share their work when they
- trust you,
- when they can get something out of it (a publication/free consulting/interesting contacts with future collaborators),
- when they know you’re not an amateur, or
- when they are promised that you won’t publish any corrections without them looking at it first.
I personally would hesitate to promise the latter (authors pre-viewing your results) to be honest.
You have probably already told them that you are not replicating to find errors, but to build on their work and generate policy advice. If they are still hesitant, I would try to:
- Establish a personal connection via a personal visit or skype to discuss the research (and with that, the data). This is my best tip.
- You may want to offer them a collaborative approach.
- Point out that you are working with a more senior supervisor/thesis advisor (give name; potentially ask them to get in touch with the authors) and within your PhD program. This will show the authors that you are not an amateur who may damage their reputation with a badly done replication; but that there is an institution and a more senior advisor backing you up and making sure that you aren’t doing any nonsense.
I (and my students) have written lots of nice, non-threatening emails to authors to ask for data – but when they don’t really know you or haven’t seen your face e.g. in person or on skype, it may be difficult to establish trust.
Tenopir C, Allard S, Douglass K, Aydinoglu AU, Wu L, et al. (2011) Data Sharing by Scientists: Practices and Perceptions. PLoS ONE 6(6): e21101. doi:10.1371/journal.pone.0021101
Lupia, A., & Elman, C. (2014). Openness in political science: Data access and research transparency. PS:Political Science & Politics, 47(01), 19-42.