Kavanaugh: A Probability Poll

There’s some literature suggesting that people are more careful when they think in probabilities. If you ask them for a definite answer, they might give it and sound very confident, but if you encourage them to think probabilistically they might admit there’s more uncertainty.

I wanted to look into this in the context of the recent Supreme Court confirmation hearings, so I asked readers to estimate their probability that Judge Kavanaugh was guilty of sexually assaulting Dr. Ford. I got 2,350 responses (thank you, you are great). Here was the overall distribution of probabilities. Horizontal axis is percent chance he did it; vertical axis is number of people who responded with that percent:

This looks weird because people were most likely to give numbers rounded off the the nearest ten.

I separated responses into bins from 0 – 9%, 10 – 19%, and so on to 90 – 100%. Keep in mind that the last bin is slightly larger than the others, so it might make it unfairly look like more people gave extreme high answers than extreme low answers. I also switched the vertical axis to percent of responses in each bin. Smoothed out, it looks like this:

This looks pretty balanced, and it is: the average probability is 52.64%. This is probably a fake balance based on all the different demographic skews involved cancelling out: 2.5x as many Democrats as Republicans answered the survey, but 9x as many men as women did.