A group of 625 political scientists signed a petition to delay the new APSA guidelines for transparency. They want to discuss the implications for qualitative data, hand-written field notes and confidential data first. I agree that practical discussions are necessary – but this should not be a reason to abandon the transparency guidelines.
In 2014, the American Political Science Association issued the “Data Access and Research Transparency (DA-RT)” guidelines for more transparency in the discipline. The guidelines state that data and code should be made available when publishing in a journal. So far, the guidelines have been adopted by 27 journals [read the full guidelines here].
Why a transparency backlash?
Now there’s a transparency backlash. In a “Petition to Delay DA-RT Implementation” [see pdf], 625 political scientists fear that some researchers will not be able publish their work when they cannot provide the data – especially when they work with sensitive and qualitative information. The petition states:
[T]he decision about whether or not to make research materials publicly available should belong to scholars and not journal editors or reviewers.
The signatories worry that “many key questions remain unresolved”. For example:
- How to achieve transparency for non-statistical research
- Who covers costs involved with preparing data for archiving (especially hand-written data)
- Legal and ethical obligations to human subjects (e.g. in casese of research on political violence)
- Will a data repository provide graded access, and is it safe to hacking
In order to discuss these issues, the petition requests that journals abandon the transparency guidelines until at least after the 2016 annual meeting.
Why we should not delay the transparency guidelines
Of course, the concerns are important and valid. I agree that we should be very clear about what kind of information should be made transparent, and in which cases that is not possible. The DA-RT guidelines also acknowledge that:
6.4 Scholars may be exempted from Data Access and Production Transparency in order to
(A) address well-founded privacy and confidentiality concerns, including abiding by relevant human subjects regulation; and/or
(B) comply with relevant and applicable laws, including copyright.
Decisions to withhold data and a full account of the procedures used to collect or generate them should be made in good faith and on reasonable grounds.
6.7 Nothing in this section shall require researchers to transfer ownership or other proprietary rights they may have.
6.8 As citizens, researchers have an obligation to cooperate with grand juries, other law enforcement agencies, and institutional officials. Conversely, researchers also have a professional duty not to divulge the identity of confidential sources of information or data developed in the course of research, whether to governmental or nongovernmental officials or bodies, even though in the present state of American law they run the risk of suffering an applicable penalty.
We should discuss the practicality of these points in detail. But it would be a grave mistake to delay the guidelines.
First, it is simply not necessary. The guidelines have inbuilt exemptions so that no journal editor will have reason to reject a paper that uses sensitive data.
Second, stopping the current guidelines can actually hinder the overall progress in reproducibility we have made in the discipline. It has taken years to agree on these guidelines. Many other disciplines have already made much more progressive suggestions in replication of published work, reproducibility and data transparency. Stopping the DA-RT guidelines now will not solve any of the issues discussed above, but it may prevent us from finding a solution to irreproducible articles and authors withholding data for no good reason.
Third, we would send a very questionable signal to the other disciplines if we abandon the new guidelines now. Interdisciplinary discussions (see Royal Society, TOP guidelines, Reproducibility Project) show that most fields agree that any scholar who wants his work to be legitimate must make the sources, procedures and data – as much as possible – available. Universities and funders in most disciplines have acknowledged that.
If details about research findings are not made available, scholars cannot evaluate and cross-check published work. Just have a look at the recent LaCour scandal in political science, where a scholar tried to cover up data fabrication in his highly influential paper.
What should we do?
We should keep the existing DA-RT guidelines intact, and work on creating further practical implementation guidelines, check-lists and ‘how-to’ manuals.
This could be a start:
- How to achieve transparency for non-statistical forms of inquiry: There are two good articles on qualitative data, “Data Access and Research Transparency in the Qualitative Tradition” and “Transparency: The Revolution in Qualitative Research“
- Who covers costs involved with preparing data for archiving (especially when raw data are not easily digitalized): Many funders now require transparency, and this can be written into the budget. Data archives are for free.
- Legal and ethical obligations to human subjects: These are exempt from the transparency rule.
- Will a data repository provide graded access, and is it safe to hacking: Data can be made available in repositories such as the Institution for Social and Policy Studies (ISPS) at Yale University, the Dataverse Network at Harvard University, and the Inter-university Consortium for Political and Social Research (ICPSR), as a Open Science Framework project page, on journal-specific Web sites and archives; or on the original author’s webpages. The Harvard Dataverse allows a setting where access to particular files can be blocked. The author can decide whom to allow to download the files. The OSF project pages also allow to distinguish between public and blocked files.