This week thousands of researchers used the hashtag #OverlyHonestMethods to admit how they or their colleagues manipulated data, gave up on experiments, included nonsense because a reviewer requested it. These confessions are hilarious, and a bit unsettling. Here’s a selection of tweets interesting for political and social scientists. After reading those, we don’t have to wonder anymore why no one provides their replication data. The main themes right now for social and political scientists seem to be: didn’t read citations; pleasing reviewers; statistics manipulation; please don’t replicate my study; qualitative and mixed methods issues; my supervisor told me so; forging significance; I don’t know what I’m doing.
Tweets about replication and reproducibility
Since the first #overlyhonestmethods hash tag was used on January 7, 23 tweets include concerns or jokes about reproducibility. I summarize them here. Tweets including “replication” are here.
Selected posts relevant for political and social scientists
Political Science Replication 
I just wrote something about this too (maybe more from a biological/health science point of view). Thanks for sharing the collection.
LikeLike
Thank you, I will have a look!
LikeLike
Hi, “Bookscout” here and I would like to unpack my tweet a bit.
My jab was at review articles and how they tend to be used as a reference instead of people bothering to track ideas to the origin. Fair enough, but a substantial review with 200 references is an enormous amount of labor to create – typical scientists read (and by this I mean read every bit, not skim) about 250-300 articles a year (I believe this is from a Tenopir/King paper). Lack of time and need for a concise summary is why reviews exist (and perhaps as a way to get easy citations for young scholars).
So it’s a bit lazy to cite a review, but social scientists are pretty happy to cite a well-written academic book instead of going to the primary texts on which they are based, or the many journal articles that get cited along the way in a book. I’m a librarian now, so citation analysis and the gap between what influences and what is actually cited is one of my personal hobbyhorses.
Your overall mission – more replication – is truly noble. I wish you the best of luck.
Ian McCullough, “Bookscout”
LikeLike
[…] evidence (see Retraction Watch). After the twitter hype about misconduct in the academic world (#overlyhonestmethods) the discussion on replication, reproducibility and data sharing is attracting even more […]
LikeLike
Reblogged this on Make Coffee Not War and commented:
This makes me feel better about my masters thesis. Apparently, not many of us know what we’re doing.
LikeLike
It helps to reinforce the idea tha social science are a fake. Thanks! What about this practices in the straight “Sciences” (with uppercase)? What do you do when you are not defaming social sciences? I´m a llttle tired of this bullshit
LikeLike
[…] early January, the twitter hashtag #overlyhonestmethods revealed the current state of replication and reproducibility in the sciences. Thousands of […]
LikeLike