#overlyhonestmethods analyzed: tweets about replication

Screen Shot 2013-01-10 at 11.47.18 PMI’m collecting #overlyhonestmethods tweets about reproducibility and replication. For me, those two terms describe the same issue. Whenever data were manipulated, and files or code are not available online, we can’t check the results. They are not replicable or reproducible. Below are the tweets about replication. The main themes seem to be that authors don’t want others to replicate (and check) their work; that some studies are simply not replicable; and that the community does not value replication enough (or fund it properly).

[Update: Now including replication tweets from Jan 9 to Jan 11. There are some fun ones among them!]

Replication vs. Reproducibility

I do not agree with the distinction between “replication” and “reproducibility” – at least for political science these terms mean the same. In some fields, “reproducible” means that someone go to the same results when using the same data set, while “full replication doesn’t analyze the same dataset, but rather involves an independent investigator collecting an independent dataset conducting an independent analysis” (see Simply Statistics Blog). Obviously, it is harder to achieve full replication since data sets in political science (from the UN, World Bank etc.) are continuously changing, being updated, revised etc. Most political scientists use replication and reproducibility as synonyms. The main issue is if you are able to get he same results with the same data set.

While I reported my search for “reproducibility” separately (for practical reasons), here are tweets about replication.

Tweets about replication Batch II (Jan 9-11)

Screen Shot 2013-01-11 at 9.39.22 AM

Screen Shot 2013-01-11 at 9.36.38 AM

Screen Shot 2013-01-11 at 9.37.05 AM

Screen Shot 2013-01-11 at 9.38.12 AM

Screen Shot 2013-01-11 at 9.40.35 AM

Screen Shot 2013-01-11 at 9.41.02 AM

Screen Shot 2013-01-11 at 9.43.44 AM

Screen Shot 2013-01-11 at 9.45.38 AM

Some extra gems

These slipped through because they don’t include my search words. They say a lot about replication and reproducibility practice.

Screen Shot 2013-01-11 at 6.27.30 PM

Screen Shot 2013-01-11 at 6.27.42 PM

Screen Shot 2013-01-11 at 6.22.02 PM

Tweets about replication Batch I (Jan 7-9)

Screen Shot 2013-01-10 at 10.19.38 PM

Screen Shot 2013-01-10 at 10.21.23 PM

Screen Shot 2013-01-10 at 10.21.55 PM

Screen Shot 2013-01-10 at 10.23.38 PM

Screen Shot 2013-01-10 at 10.24.10 PM

Screen Shot 2013-01-10 at 11.06.11 PM

Screen Shot 2013-01-10 at 11.06.56 PM

Screen Shot 2013-01-10 at 11.08.00 PM

Screen Shot 2013-01-10 at 11.09.10 PM

Screen Shot 2013-01-10 at 11.14.16 PM

About this tweet list

I included tweets that came up when searching for replicat*.

  • Batch I includes tweets between Day 1 (January 7, first tweet by dr_leigh) to January 9, 5.17pm.
  • Batch II includes tweets from Jan 9 (where I left off) to Jan 11 (8.55am).

I mostly excluded tweets discussing biological replicates/ replications of experiments, because this blog is more about replication of existing articles to check the validity of their results (also called reproducibility). This is a selection of replication tweets which might not include exactly all tweets on the topic.

Advertisements
Tagged , , , , , , ,

4 thoughts on “#overlyhonestmethods analyzed: tweets about replication

  1. […] the hash tag #overlyhonestmethods went viral, I have been checking tweets for reproducibility and replication issues. Many tweets are in some way connected: every time a researcher admits being […]

    Like

  2. […] Since the first #overlyhonestmethods hash tag was used on January 7, 23 tweets include concerns or jokes about reproducibility. I summarize them here. Tweets including “replication” are here. […]

    Like

  3. […] have recently seen a range of ‘confessions’ of natural and social scientists on twitter (#overlyhonestmethods). Some of them joke “you will never be able to reproduce our […]

    Like

  4. […] supervisor told me so”; I don’t know what I’m doing! I collected the main tweets about replication and […]

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: