#73 A Guide to Plotting Inferences & Uncertainties of Bayesian Models, with Jessica Hullman

Proudly sponsored by PyMC Labs, the Bayesian Consultancy. Book a call, or get in touch!

I’m guessing you already tried to communicate the results of a statistical model to non-stats people — it’s hard, right? I’ll be honest: sometimes, I even prefer to take notes during meetings than doing that… But shhh, that’s out secret.

But all of this was before. Before I talked with Jessica Hullman. Jessica is the Ginny Rometty associate professor of computer science at Northwestern University.

Her work revolves around how to design interfaces to help people draw inductive inferences from data. Her research has explored how to best align data-driven interfaces and representations of uncertainty with human reasoning capabilities, which is what we’ll mainly talk about in this episode.

Jessica also tries to understand the role of interactive analysis across different stages of a statistical workflow, and how to evaluate data visualization interfaces.

Her work has been awarded with multiple best paper and honorable mention awards, and she frequently speaks and blogs on topics related to visualization and reasoning about uncertainty — as usual, you’ll find the links in the show notes.

Our theme music is « Good Bayesian », by Baba Brinkman (feat MC Lars and Mega Ran). Check out his awesome work at https://bababrinkman.com/ !

Thank you to my Patrons for making this episode possible!

Yusuke Saito, Avi Bryant, Ero Carrera, Giuliano Cruz, Tim Gasser, James Wade, Tradd Salvo, Adam Bartonicek, William Benton, James Ahloy, Robin Taylor, Thomas Wiecki, Chad Scherrer, Nathaniel Neitzke, Zwelithini Tunyiswa, Elea McDonnell Feit, Bert≈rand Wilden, James Thompson, Stephen Oates, Gian Luca Di Tanna, Jack Wells, Matthew Maldonado, Ian Costley, Ally Salim, Larry Gill, Joshua Duncan, Ian Moran, Paul Oreto, Colin Caprani, Colin Carroll, Nathaniel Burbank, Michael Osthege, Rémi Louf, Clive Edelsten, Henri Wallen, Hugo Botha, Vinh Nguyen, Raul Maldonado, Marcin Elantkowski, Adam C. Smith, Will Kurt, Andrew Moskowitz, Hector Munoz, Marco Gorelli, Simon Kessell, Bradley Rode, Patrick Kelley, Rick Anderson, Casper de Bruin, Philippe Labonde, Michael Hankin, Cameron Smith, Tomáš Frýda, Ryan Wesslen, Andreas Netti, Riley King, Yoshiyuki Hamajima, Sven De Maeyer, Michael DeCrescenzo, Fergal M, Mason Yahr, Naoya Kanai, Steven Rowland, Aubrey Clayton, Jeannine Sue, Omri Har Shemesh, Scott Anthony Robson, David Haas, Robert Yolken, Or Duek, Pavel Dusek, Paul Cox and Trey Causey.

Visit https://www.patreon.com/learnbayesstats to unlock exclusive Bayesian swag ;)

General links from the show:

• Jessica’s website: http://users.eecs.northwestern.edu/~jhullman/ 
• Jessica on Twitter: https://twitter.com/JessicaHullman
• Midwest Uncertainty Collective: https://mucollective.northwestern.edu/
• Jessica’s posts on Andrew Gelman’s blog: https://statmodeling.stat.columbia.edu/
• Jessica’s posts on Medium: https://medium.com/multiple-views-visualization-research-explained
• LBS # 66, Uncertainty Visualization & Usable Stats, with Matthew Kay: https://learnbayesstats.com/episode/66-uncertainty-visualization-usable-stats-matthew-kay/

Some of Jessica’s research that she mentioned:

• A Bayesian Model of Cognition to Improve Data Visualization: https://mucollective.northwestern.edu/files/2019-BayesianVis-CHI.pdf
• Visual Reasoning Strategies for Effect Size Judgments and Decisions: https://mucollective.northwestern.edu/files/2020%20-%20Kale,%20Visual%20Reasoning%20Strategies%20for%20Effect%20Size%20Judgements.pdf
• Hypothetical Outcome Plots Help Untrained Observers Judge Trends in Ambiguous Data: https://mucollective.northwestern.edu/files/2018-HOPsTrends-InfoVis.pdf

Behavioral economics paper Jessica mentioned:

• A Model of Non-belief in the Law of Large Numbers: https://scholar.harvard.edu/files/rabin/files/barney2014.pdf

More on David Blackwell:

• Summary of his career: https://stat.illinois.edu/news/2020-07-17/david-h-blackwell-profile-inspiration-and-perseverance
• His original work on Blackwell ordering: https://projecteuclid.org/journals/annals-of-mathematical-statistics/volume-24/issue-2/Equivalent-Comparisons-of-Experiments/10.1214/aoms/1177729032.pdf
• Lectures on day 5 of this workshop covered his work on approachability: https://old.simons.berkeley.edu/workshops/schedule/16924

Abstract:

by Christoph Bamberg

Professor Jessica Hullman from Northwestern University is an expert in designing visualisations that help people learn from data and not fall prey to biases.

She focuses on the proper communication of uncertainty, both theoretically and empirically.

She addresses questions like “Can a Bayesian model of reasoning explain apparently biased reasoning?”, “What kind of visualisation guides readers best to a valid inference?”, “How can biased reasoning be so prevalent - are there scenarios where not following the canonical reasoning steps is optimal?”.

In this episode we talk about her experimental studies on communication of uncertainty through visualisation, in what scenarios it may not be optimal to focus too much on uncertainty and how we can design models of reasoning that can explain actual behaviour and not discard it as biased.