“People like stories” A short film about reproducibility

janz replication spiegelhalterWe need mathematical help to tell the difference between a real discovery and the illusion of one. Fellow of the Royal Society and future President of the Royal Statistical Society, Sir David Spiegelhalter visits Dr Nicole Janz  to discuss reproducibility in scientific publications. Here’s the film:

Continue reading

Advertisements

Published articles understate the probability of type I errors

Screen Shot 2015-12-16 at 17.52.20Can we trust published articles in political science? A recent paper suggests that we should be sceptic. When comparing the published results of survey experiments with the pre-registered plans for the same study, a lot of information gets lost. 80 percent of the studies failed to report all experimental conditions and planned outcomes.

Continue reading

Tagged , , ,

How Reporting Statistical Significance Causes p-Value Hacking

P-value hacking Nicole Janz Replication BlogA new article by researchers at the University of Amsterdam shows that publication bias towards statistically significant results may cause p-value misreporting. The team examined hundreds of published articles and found that authors had reported p-values < .05 when they were in fact larger. They conclude that publication bias may incentivize researchers to misreport results.

Continue reading

Tagged ,

Re-Blog: Who’s afraid of open data? (by Dorothy Bishop on the BishopBlog)

“Well, how would you like it if you had spent years of your life gathering data … and some person you have never heard comes out of nowhere demanding to have it?” Dorothy Bishop wrote a very interesting blog post about a conference chat about someone who was forced to work transparently. Here’s the first few paragraphs – do check out the full text:
Continue reading

Political Scientists Trying to Delay Research Transparency

A group of 625 political scientists signed a petition to delay the new APSA guidelines for transparency. They want to discuss the implications for qualitative data, hand-written field notes and confidential data first. I agree that practical discussions are necessary – but this should not be a reason to abandon the transparency guidelines.
Continue reading

Tagged , , ,

Does transparency hinder novelty and creativity?

The journal Epidemiology has refused to support transparency guidelines. In its editorial it gives the reason: research transparency hinders novelty and creativity.

Continue reading

Guest Post: The Replication Road – Scientific Detour or Destination? By Nate Breznau

Guest Post: I asked Nate Breznau, an empirical sociologist, to write about his experience replicating published work – the challenges, benefits, and how he got published.

Continue reading

Tagged , ,