MCMC and Gibbs sampling

Content

  • Metropolis-adjusted Langevin algorithm (MALA)
  • Gibbs sampling
  • Data augmentation
  • Graphical diagnostics of convergence
  • Goodness-of-fit measures (WAIC, LOO-CV, etc.)

Learning objectives

At the end of the chapter, students should be able to

  • implement Gibbs sampling
  • derive the conditional distributions of a model for Gibbs sampling
  • choose suitable test statistics to evaluate model adequacy
  • assess convergence using graphical tools and effective sample size
  • perform model comparisons using Bayes factor or predictive measures

Readings

Warning

These readings should be completed before class, to ensure timely understanding and let us discuss the concepts together through various examples and case studies — the strict minimum being the course notes.

Complementary readings

Warning

Complementary readings are additional sources of information that are not required readings, but may be useful substitutes. Sometimes, they go beyond the scope of what we cover and provide more details.

  • Gelman et al. (2013), chapters 6 and 7
  • Albert (2009), chapters 6 and 10 (several examples)
  • McElreath (2020), chapter 9.5

Slides

 View all slides in new window  Download PDF of all slides

Code

References

Albert, J. (2009). Bayesian computation with R (2nd ed.). Springer. https://doi.org/10.1007/978-0-387-92298-0
Gelman, A., Carlin, J. B., Stern, H. S., Dunson, D. B., Vehtari, A., & Rubin, D. B. (2013). Bayesian data analysis (3rd ed.). Chapman; Hall/CRC. https://doi.org/10.1201/b16018
McElreath, R. (2020). Statistical rethinking: A Bayesian course with examples in R and STAN (2nd ed.). Chapman; Hall/CRC.