Ryan Seals August 12, 2017 A p-Value Problem? Ryan Seals August 12, 2017 There's been a lot of hand-wringing over the past few years about a "crisis of reproducibility" in science. A recent paper in Nature has been getting attention for proposing a simple solution: Marcus Munafo, a professor of experimental psychology at the University of Bristol and one of the authors of the paper, told BuzzFeed News: “We’re past the point where we can just highlight the problem and say how terrible it is. We need to think about ways in which we can improve the quality of what we do.” Their suggestion is to make it much harder to declare that you’ve found a “statistically significant” result in the first place. I think this is misguided, for the reasons discussed in this article from Slate: But more broadly, [Dr. Deborah Mayo] was skeptical of the idea that lowering the informal p-value threshold will help fix the problem, because she’s doubtful such a move will address “what almost everyone knows is the real cause of nonreproducibility”: the cherry-picking of subjects, testing hypothesis after hypothesis until one of them is proven correct, and selective reporting of results and methodology.