Getting started with Bayesian statistics

From time to time I receive a question from students and collaborators how to get started with Bayesian statistics. So here it comes: the list of my favourite resources!

Getting started

• Take any two examples from PyMC Gallery. For example, GLM and hierarchical partial pooling. Implement them in a Jupyter notebook. Rather than copying and running the code, type it on your own and think what it is supposed to do.
• Read Michael Betancourt’s Inferring gravity from data. Reproduce it in PyMC — the data are available on GitHub.
• If you would like a lecture series to watch, there’s Richard McElreath’s Statistical rethinking.

Consult regularly

• Learning Bayesian Statistics is a truly excellent podcast hosted by Alexandre Andorra. Many leading statisticians, including Frank Harrell, Jessica Hullman, Kevin Murphy and Aki Vehtari. I find it the best place to learn about various perspectives on modelling techniques and important problems people are working on. I also enjoy listening to Data and Science with Glen Wright Colopy, which covers a wide range of topics and featured many prominent statisticians, such as Deborah Mayo, Chris Holmes, Andrew Gelman and David Dunson.
• Andrew Gelman’s blog and Frank Harrell’s blog.
• And, of course, whenever I open Bayesian data analysis, Probabilistic machine learning and Michael Betancourt’s notes, I learn something new.

Handbooks

General references, covering many topics

• Bayesian Data Analysis from Andrew Gelman et al.
• Biostatistics for Biomedical Research written by Frank Harrell.
• Michael Betancourt’s notes.
• Probabilistic Machine Learning written by Kevin Murphy.

Principled modelling workflow

• The Bayesian workflow manuscript.
• Michael Betancourt’s workflow description.
• Kris Sankaran and Susan Holmes wrote a great paper Generative models: an interdisciplinary perspective.
• There’s David Blei’s Build, compute, critique, repeat paper.

Inference methods

• Variational inference: a review for statisticians.
• Handbook of Markov chain Monte Carlo (unfortunately, this book is not freely available).
• An introduction to sequential Monte Carlo (unfortunately, this book is not freely available).

Great to watch

If you are in mood for something as engaging as a TV series episode, but more informative, take a look at these talks:

• David Dunson’s Debunking the hype.
• Kristin Lennox’s Everything wrong with statistics.
• Andrew Gelman’s Solve all your statistics problems using p-values.