Reproducibility in qualitative research is not an accepted standard yet. While some researchers try to be transparent and might provide interview transcripts on request, most qualitative work is seldom checked and validated. I talked to political scientist Ingo Rohlfing, University of Cologne, about why reproducibility should not just be an issue for quantitative methods, and how case study researchers can make their data available.
Are replication and reproducibility issues that only statistical researchers have to deal with?
No, qualitative and quantitative researchers should both ensure replication and reproducibility.
I feel that a lot of qualitative researchers remain reluctant to the idea of publishing their data. Would you agree that this is the case, and why could that be?
I wouldn’t say qualitative researchers are reluctant. In my view, the point is more that replication is not an issue in qualitative research, at least not at present. Except field research and the publication of field notes, there is little awareness of and discussion about making qualitative data available.
Why would reproducibility be important also for qualitative work?
For the same reason it is important in quantitative work: We need to know the evidence on the basis of which qualitative researchers arrive at their inferences.
How could a researcher validate a published qualitative study that is based on narratives, interviews or text analysis?
Should we just ‘trust’ qualitative researchers and their interpretations more than quantitative researchers?
Ex ante, there is no reason to trust qualitative researchers more or less than quantitative researchers.
Could you think of ways how qualitative researchers can try to be transparent, e.g. by posting their interview transcripts or documents on a dataverse?
That’s exactly what they should do: publish the sources PLUS the codings, e.g. what interview statements are taken as evidence for a specific inference.
Would it be too cost-intensive to publish field notes, transcripts etc?
I don’t think so. It would be time-consuming, but not necessarily cost-intensive (unless time is money).
Is there a dataverse or a platform that could work for handling vast amounts of qualitative data?
I guess the Dataverse Network would be able to handle this.
What kind of ethical issues come into play when publishing qualitative data?
Some sources may be confidential, e.g. interview partners want to stay anonymous.
What kind of material and data do you work with personally?
In my case studies, I used primary and secondary sources. At present, I am working with quantitative data.
Do you make your data available, and in which form?
The quantitative data is made available via the Dataverse Network.
Ingo Rohlfing answered these questions noting: “I assume that I am writing about qualitative research accepting the idea of replication; some variants of qualitative research conceive of qualitative research as subjective and non-reproducible.”
Ingo Rohlfing is an Assistant Professor for Comparative Social Research at the Cologne Graduate School. His main research projects are about the dynamics of party competition and on parties as organizations. In addition, he is working on social science methodology with an emphasis on case studies, multi-method research, and philosophy of science concerned with causation and causal inference. Ingo Rohlfing published the book Case Studies and Causal Inference. http://ingorohlfing.wordpress.com/ | @Ingorohlfing
The interview was conducted via email on February 1, 2013.
More on qualitative work & reproducibility
Read Todd Landman’s interview who talks about qualitative research and replication as well.