Category Archives: replication fun

Is there a difference between replication, reproduction, and re-analysis?

In my replication workshop I’ve been discussing with grad students what replication actually is. Some students, from the field of Psychology, said that it involves re-doing data collection from scratch. In political science, I feel reproducing a paper is the same as replicating work: you take the data set, re-run the analysis, and potentially add to that. Here’s a selection of definitions.
Continue reading

Advertisements
Tagged , , , , , , ,

Big Bang Theory and how Leonard ‘just’ replicates work

bb-headThere’s a funny scene in Big Bang Theory’s Episode 15 (Season 2). Leonard’s mother Dr Beverley Hofstadter is visiting him at his lab, and she’s not impressed by his replication work.
Continue reading

Tagged , , , , ,

#overlyhonestmethods analyzed: tweets about replication

Screen Shot 2013-01-10 at 11.47.18 PMI’m collecting #overlyhonestmethods tweets about reproducibility and replication. For me, those two terms describe the same issue. Whenever data were manipulated, and files or code are not available online, we can’t check the results. They are not replicable or reproducible. Below are the tweets about replication. The main themes seem to be that authors don’t want others to replicate (and check) their work; that some studies are simply not replicable; and that the community does not value replication enough (or fund it properly).
Continue reading

Tagged , , , , , , ,

#overlyhonestmethods analyzed: tweets about reproducibility

Screen Shot 2013-01-10 at 11.44.00 PMSince the hash tag #overlyhonestmethods went viral, I have been checking tweets for reproducibility and replication issues. Many tweets are in some way connected: every time a researcher admits being ‘creative’ about getting significant results or slightly ‘polishing’ tables and figures, it will be harder to check or replicate their results. Some tweets, however, directly refer reproducible results. Here they are:
Continue reading

Tagged , , , , , , , ,

#overlyhonestmethods Best tweets about tricks and manipulation in social sciences

Screen Shot 2013-01-10 at 11.52.23 PM
This week thousands of researchers used the hashtag #OverlyHonestMethods to admit how they or their colleagues manipulated data, gave up on experiments, included nonsense because a reviewer requested it. These confessions are hilarious, and a bit unsettling. Here’s a selection of tweets interesting for political and social scientists. After reading those, we don’t have to wonder anymore why no one provides their replication data. The main themes right now for social and political scientists seem to be: didn’t read citations; pleasing reviewers; statistics manipulation; please don’t replicate my study; qualitative and mixed methods issues; my supervisor told me so; forging significance; I don’t know what I’m doing.
Continue reading

Tagged , , , , , , , ,