Category Archives: publication bias

Published articles understate the probability of type I errors

Screen Shot 2015-12-16 at 17.52.20Can we trust published articles in political science? A recent paper suggests that we should be sceptic. When comparing the published results of survey experiments with the pre-registered plans for the same study, a lot of information gets lost. 80 percent of the studies failed to report all experimental conditions and planned outcomes.

Continue reading

Tagged , , ,

How Reporting Statistical Significance Causes p-Value Hacking

P-value hacking Nicole Janz Replication BlogA new article by researchers at the University of Amsterdam shows that publication bias towards statistically significant results may cause p-value misreporting. The team examined hundreds of published articles and found that authors had reported p-values < .05 when they were in fact larger. They conclude that publication bias may incentivize researchers to misreport results.

Continue reading

Tagged ,

Guest Post: The Replication Paradox, by Michèle B. Nuijten

Will integrating original studies and published replications always improve the reliability of your results? No! Replication studies suffer from the same publication bias as original studies. In her guest post, Michèle B. Nuijten, who focuses on statistical errors and data manipulation in psychology, presents two solutions to this problem.
Continue reading

Tagged , , , , , ,