Reproducibility in qualitative research: “There is little awareness”

rohlfing-head
Reproducibility in qualitative research is not an accepted standard yet. While some researchers try to be transparent and might provide interview transcripts on request, most qualitative work is seldom checked and validated. I talked to political scientist Ingo Rohlfing, University of Cologne, about why reproducibility should not just be an issue for quantitative methods, and how case study researchers can make their data available.


ingorohlfingAre replication and reproducibility issues that only statistical researchers have to deal with?

No, qualitative and quantitative researchers should both ensure replication and reproducibility.


I feel that a lot of qualitative researchers remain reluctant to the idea of publishing their data. Would you agree that this is the case, and why could that be?

I wouldn’t say qualitative researchers are reluctant. In my view, the point is more that replication is not an issue in qualitative research, at least not at present. Except field research and the publication of field notes, there is little awareness of and discussion about making qualitative data available.

Why would reproducibility be important also for qualitative work?

For the same reason it is important in quantitative work: We need to know the evidence on the basis of which qualitative researchers arrive at their inferences.

How could a researcher validate a published qualitative study that is based on narratives, interviews or text analysis?

One could use software such as MAXQDA or N-Vivo to analyze sources and code the data. As long as no confidentiality and copyright issues are involved, the codings could be published online.

Should we just ‘trust’ qualitative researchers and their interpretations more than quantitative researchers?

Ex ante, there is no reason to trust qualitative researchers more or less than quantitative researchers.

transcriptCould you think of ways how qualitative researchers can try to be transparent, e.g. by posting their interview transcripts or documents on a dataverse?

That’s exactly what they should do: publish the sources PLUS the codings, e.g. what interview statements are taken as evidence for a specific inference.

Would it be too cost-intensive to publish field notes, transcripts etc?

I don’t think so. It would be time-consuming, but not necessarily cost-intensive (unless time is money).

Is there a dataverse or a platform that could work for handling vast amounts of qualitative data?

I guess the Dataverse Network would be able to handle this.

What kind of ethical issues come into play when publishing qualitative data?

Some sources may be confidential, e.g. interview partners want to stay anonymous.

What kind of material and data do you work with personally?

In my case studies, I used primary and secondary sources. At present, I am working with quantitative data.

Do you make your data available, and in which form?

The quantitative data is made available via the Dataverse Network.

Note

Ingo Rohlfing answered these questions noting: “I assume that I am writing about qualitative research accepting the idea of replication; some variants of qualitative research conceive of qualitative research as subjective and non-reproducible.”

Bio

Ingo Rohlfing is an Assistant Professor for Comparative Social Research at the Cologne Graduate School. His main research projects are about the dynamics of party competition and on parties as organizations. In addition, he is working on social science methodology with an emphasis on case studies, multi-method research, and philosophy of science concerned with causation and causal inference. Ingo Rohlfing published the book Case Studies and Causal Inference. http://ingorohlfing.wordpress.com/ | @Ingorohlfing

Interview

The interview was conducted via email on February 1, 2013.

More on qualitative work & reproducibility

Read Todd Landman’s interview who talks about qualitative research and replication as well.

rohlfingbook

Advertisements
Tagged , , , , ,

2 thoughts on “Reproducibility in qualitative research: “There is little awareness”

  1. […] can be made available for replication. I have been advocating this by interviewing Todd Landman and Ingo Rohlfing on my blog. It suddenly occurred to me that I don’t have my own qualitative data published. […]

    Like

  2. […] on request, most qualitative work is seldom checked and validated. I talked to political scientists Ingo Rolfing and Todd Landman about how qualitative researchers can be more transparent. The importance of this […]

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: