I’m collecting #overlyhonestmethods tweets about reproducibility and replication. For me, those two terms describe the same issue. Whenever data were manipulated, and files or code are not available online, we can’t check the results. They are not replicable or reproducible. Below are the tweets about replication. The main themes seem to be that authors don’t want others to replicate (and check) their work; that some studies are simply not replicable; and that the community does not value replication enough (or fund it properly).
[Update: Now including replication tweets from Jan 9 to Jan 11. There are some fun ones among them!]
Replication vs. Reproducibility
I do not agree with the distinction between “replication” and “reproducibility” – at least for political science these terms mean the same. In some fields, “reproducible” means that someone go to the same results when using the same data set, while “full replication doesn’t analyze the same dataset, but rather involves an independent investigator collecting an independent dataset conducting an independent analysis” (see Simply Statistics Blog). Obviously, it is harder to achieve full replication since data sets in political science (from the UN, World Bank etc.) are continuously changing, being updated, revised etc. Most political scientists use replication and reproducibility as synonyms. The main issue is if you are able to get he same results with the same data set.
While I reported my search for “reproducibility” separately (for practical reasons), here are tweets about replication.
Tweets about replication Batch II (Jan 9-11)
Some extra gems
These slipped through because they don’t include my search words. They say a lot about replication and reproducibility practice.
Tweets about replication Batch I (Jan 7-9)
About this tweet list
I included tweets that came up when searching for replicat*.
- Batch I includes tweets between Day 1 (January 7, first tweet by dr_leigh) to January 9, 5.17pm.
- Batch II includes tweets from Jan 9 (where I left off) to Jan 11 (8.55am).
I mostly excluded tweets discussing biological replicates/ replications of experiments, because this blog is more about replication of existing articles to check the validity of their results (also called reproducibility). This is a selection of replication tweets which might not include exactly all tweets on the topic.