Significant Feelings

Everybody's talking about p-values. An important part of the replicability discussion has been about the correct use and interpretation of p-values, and the potentially distorting incentives attached to getting one below .05. And recently, Simine was a co-author on a paper proposing to redefine the interpretive threshold for calling something "significant." In this episode we talk about p-values: our feelings about them, how we were taught to think about them and how that has changed over the years, and the role of thresholds and categorization of evidence in our scientific thinking. Plus: A letter-writer asks if you should put more faith in the "pre-crisis" early work of present-day open science advocates.

Discussed in this episode:

• Shiny app showing the distribution of p-values, by Kristoffer Magnusson
• Redefine Statistical Significance by Benjamin et al.
• And the responses: Justify Your Alpha by Lakens et al., and Abandon Statistical Significance by McShane et al.

The Black Goat is hosted by Sanjay Srivastava, Alexa Tullett, and Simine Vazire. Find us on the web at www.theblackgoatpodcast.com, on Twitter at @blackgoatpod, or on Facebook at facebook.com/blackgoatpod/. You can email us at [email protected]. You can subscribe to us on iTunes.

Our theme music is Peak Beak by Doctor Turtle, available on freemusicarchive.org under a Creative Commons noncommercial attribution license.

This is episode 22. It was recorded November 22, 2017.