Reblog: Social science and replication

My colleague Alex Sutherland (Twitter: @as2140) and I have written a short piece on replication for the Alliance for Useful Evidence Blog.

Social science and replication

The hesitation to reproduce and replicate datasets inhibits clarity and can obscure research results. Alex Sutherland and Nicole Janz argue for a cultural shift to demand more replication of results which can help policymakers and academics alike to bring greater legitimacy and relevance to research.

Social science is broken. The far-too-prevalent reporting of ‘statistically significant’ results, results ‘bordering on significance’ creeping in, focusing on significance rather than significance and effect size, undeclared conflicts of interest, weak or non-existent peer-review processes, or unwillingness to retract articles, create headaches for anyone trying to make sense of research output in social science (and science in general).

But all is not lost. … [Read the full article on the Alliance for Useful Evidence Blog, published Dec 13, 2013].

Advertisements
Tagged , ,

3 thoughts on “Reblog: Social science and replication

  1. Doug Scott says:

    In essence, aren’t we simply defining the role of a referee? After all the idea of getting a referee to read a paper prior to publication is to provide an expert opinion on the validity of the paper. We could extend the role of the referee to include replication.

    Like

    • Interesting idea! I discussed this earlier with a few people – it seems reviewers are already stressed out doing reviews at all. If they have to do a replication (for free), they might not agree to review any more. How would you tackle that problem?

      Like

      • Doug Scott says:

        Yes, that did occur to me when I was posting it. In the computer world, it’s a fundamental principle that those who devise a system cannot be trusted to verify its operation – that must be done by an independent body (different management, different reporting lines), AND…

        Until a system has been verified by an independent team, it is never allowed to see the light of day (and I think all commercially successful sites adopt this practice).

        Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: