Category Archives: pre-registration

Published articles understate the probability of type I errors

Screen Shot 2015-12-16 at 17.52.20Can we trust published articles in political science? A recent paper suggests that we should be sceptic. When comparing the published results of survey experiments with the pre-registered plans for the same study, a lot of information gets lost. 80 percent of the studies failed to report all experimental conditions and planned outcomes.

Continue reading

Advertisements
Tagged , , ,

How Reporting Statistical Significance Causes p-Value Hacking

P-value hacking Nicole Janz Replication BlogA new article by researchers at the University of Amsterdam shows that publication bias towards statistically significant results may cause p-value misreporting. The team examined hundreds of published articles and found that authors had reported p-values < .05 when they were in fact larger. They conclude that publication bias may incentivize researchers to misreport results.

Continue reading

Tagged ,

Guest Post: Six Crucial Steps to Strengthen a Replication, by Mallory Kidwell

Scientific research advances knowledge by making transparent the methodology and evidence supporting scientific claims. This way, others can evaluate the basis of evidence and repeat the research in order to confirm the conclusions. However, the current incentive structures emphasizing publication can cause a disconnect between the valued ideals and daily practices of individual researchers.
Continue reading

Tagged ,

Guest Post: The case for pre-registration in social science, by Jamie Monogan

pre-registration replicationPolitical scientist Jamie Monogan, University of Georgia, discusses preregistration: Why should you announce your data analysis beforehand? Which journals support preregistration? And where can you register your study?
Continue reading

Tagged , , ,

Guest post: Stop trusting other researchers

Guest post by L.J Zigerell: Current practice in the social sciences places trust in researchers regarding their data collection, analysis, and reporting of results. That trust is sometimes unwarranted. Instead, we should increase trust in social science by encouraging tools of reproducibility: replication studies, pre-registration, third-party data collection, and open data.
Continue reading

Tagged , , ,

Guest Post: The Replication Paradox, by Michèle B. Nuijten

Will integrating original studies and published replications always improve the reliability of your results? No! Replication studies suffer from the same publication bias as original studies. In her guest post, Michèle B. Nuijten, who focuses on statistical errors and data manipulation in psychology, presents two solutions to this problem.
Continue reading

Tagged , , , , , ,

Against publication bias: Politics journal invites pre-registered papers

After fields like Psychology have explored pre-registration of studies, now Political Science follows the trend for more transparency in research. A high ranking journal, Comparative Political Studies, plans a special issue in which all papers have to be pre-registered and are submitted without the results section. I asked Michael Findley, one of the editors behind the initative, about publication bias and transparency in the field. Continue reading

Tagged , , , , , ,