Regression models

Content

  • Bayesian linear model
  • Priors for covariance matrices
  • Shrinkage prior
  • Bayesian model averaging

Learning objectives

At the end of the chapter, students should be able to

  • code a Gaussian linear model using Gibbs sampling
  • determine suitable priors for correlated random effects
  • understand the methodology behind shrinkage priors

Readings

Complementary readings

  • Spike and slab priors: Mitchell & Beauchamp (1988), George & McCulloch (1993)
  • Bayesian linear model: Geweke (1992)
  • Horseshoe priors: Carvalho et al. (2010), Piironen & Vehtari (2017) ()
  • Comparison of shrinkage priors: Erp et al. (2019)
  • Bayesian model averaging: Hoeting et al. (1999), Holmes et al. (2002)

Slides

 View all slides in new window  Download PDF of all slides

Code

References

Carvalho, C. M., Polson, N. G., & Scott, J. G. (2010). The horseshoe estimator for sparse signals. Biometrika, 97(2), 465–480. https://doi.org/10.1093/biomet/asq017
Erp, S. van, Oberski, D. L., & Mulder, J. (2019). Shrinkage priors for Bayesian penalized regression. Journal of Mathematical Psychology, 89, 31–50. https://doi.org/10.1016/j.jmp.2018.12.004
George, E. I., & McCulloch, R. E. (1993). Variable selection via Gibbs sampling. Journal of the American Statistical Association, 88(423), 881–889. https://doi.org/10.1080/01621459.1993.10476353
Geweke, J. (1992). Evaluating the accuracy of sampling-based approaches to the calculation of posterior moments. In Bayesian statistics 4: Proceedings of the fourth valencia international meeting, dedicated to the memory of morris h. DeGroot, 1931–1989. Oxford University Press. https://doi.org/10.1093/oso/9780198522669.003.0010
Hoeting, J. A., Madigan, D., Raftery, A. E., & Volinsky, C. T. (1999). Bayesian model averaging: A tutorial. Statistical Science, 14(4), 382–401. http://www.jstor.org/stable/2676803
Holmes, C. C., Denison, D. G. T., & Mallick, B. K. (2002). Accounting for model uncertainty in seemingly unrelated regressions. Journal of Computational and Graphical Statistics, 11(3), 533–551. http://www.jstor.org/stable/1391112
Mitchell, T. J., & Beauchamp, J. J. (1988). Bayesian variable selection in linear regression. Journal of the American Statistical Association, 83(404), 1023–1032. https://doi.org/10.1080/01621459.1988.10478694
Piironen, J., & Vehtari, A. (2017). Sparsity information and regularization in the horseshoe and other shrinkage priors. Electronic Journal of Statistics, 11(2), 5018–5051. https://doi.org/10.1214/17-ejs1337si