Outline
Instructor
- Dr. Léo Belzile
- 4.850, Côte-Sainte-Catherine
- leo.belzile@hec.ca
Course details
- Winter 2025
- Monday
- 15:30-18:30
- TBD
Course content
Hands on introduction to Bayesian data analysis. The course will cover the formulation, evaluation and comparison of Bayesian models through examples.
Mathematical review. Basics of Markov Chain Monte Carlo methods and sampling algorithms, with a focus on off the shelf software (e.g., OpenBugs, Stan, INLA). Approximation methods. Hierarchical modelling, with a focus on latent Gaussian models.
Themes covered:
- Introduction to the Bayesian paradigm
- Formulation, comparison and evaluation of Bayesian models
- Sampling algorithms and Markov chain Monte Carlo methods
- Computational strategies for inference
- Hierarchical models
- Advanced topics
Target audience and prerequisites
The course is part of the PhD program in Administration offered by HEC Montréal jointly with McGill, Concordia and Université du Québec à Montréal (UQÀM). Other Québec students can register via BCI.
Course materials
I will provide slides and videos. In addition to those, there will be assigned readings from textbook and reference papers.
Textbooks
Course notes for the class can be found online
Additional references include Gelman et al. (2013), McElreath (2020) and Johnson et al. (2022).
- A. Gelman, J. Carlin, H. Stern, D. Dunson, A. Vehtari and D. Rubin (2013). Bayesian Data Analysis, 3rd edition, CRC press. doi: 10.1201/b16018
- R. McElreath (2020). Statistical Rethinking:
A Bayesian Course with Examples in R and Stan, 2nd edition, CRC Press. - A.A. Johnson, M.Q. Ott, and M. Dogucu (2022). Bayes Rules!
An Introduction to Applied Bayesian Modeling, 1st edition, CRC Press. Free available online.
Other references
There will occasionally be additional articles to read; links to these other resources will be included on the content page for that session.
Course content
Below is a tentative schedule.
Week 1: Tools of the trade
- Review of probability distributions
- Monte Carlo integration
- Likelihood based methods
- Bayes’ theorem and updating
- Marginalization and conditioning
Week 2: Basics of Bayesian inference
- Key concepts: prior, posterior and interpretation
- Predictive distributions
- Marginal likelihood and numerical integration
- Credible intervals, loss functions and posterior summaries
- The beta binomial conjugate model
Week 3: Prior beliefs
- Conjugate priors
- Flat and vague priors.
- Priors for scale parameters
- Parameter elicitation and expert knowledge
- Penalized complexity prior
- Prior sensitivity analysis
Week 4: Markov chain Monte Carlo methods
- Basics of Markov chains
- Rejection sampling
- Slice sampling
- Metropolis–Hastings algorithm
Week 5: Markov chain Monte Carlo methods
- Gibbs sampling
- Reversible jumps
- Hamiltonian Monte Carlo
Week 6: Regression models
- Bayesian generalized linear model
- Random effects and pooling
Week 7: Bayesian workflow and model diagnostics
- Bayesian workflow
- Model diagnostics (WAIC, LOO-CV, etc.)
- Estimation from Markov chain
Week 8: Computational strategies for MCMC
- Model reparametrization
- Marginalization and joint updates
- Numerical approximations
- Optimal tuning of variance parameters
Week 9: Hierarchical models
- Multi-stage modelling
- Latent Gaussian models
- Probabilistic programming
- Applications to time series and spatial data
Week 10: Deterministic approximations
- Laplace approximation
- Integrated Laplace approximation
Week 11: Variational inference
- Gradient descent
- ELBO criterion and Kullback–Leibler divergence
- Variational Bayes
Week 12: Final review
- Expectation propagation
- Model comparison and Bayes factors
- Course recap
Student hours
Monday before class or by appointment. My office, 4.850, is located next to the southern elevators in Côte-Sainte-Catherine building.
Please watch this video:
Student hours are set times dedicated to all of you (most professors call these “office hours”; I don’t1). This means that I will be in my office waiting for you to come by if you want to talk to me in person (or remotely) with whatever questions you have. This is the best and easiest way to find me and the best chance for discussing class material and concerns.
Intellectual integrity
Please don’t cheat! The official policy lists the school rules regarding plagiarism and academic integrity.
Student services
Students with special needs should feel free to approach me so we can best discuss accommodations. Do check out HEC Montréal’s disabled students and [psychological] (https://www.hec.ca/en/students/support-resources/psychological-support/index.html) support services.
Harassment and sexual violence
The Center for Harassment Intervention (BIMH) is the unique access point for all members of the community subject to harassment or sexual violence. You can reach them at 514 343-7020 or by email at harcelement@hec.ca from Monday until Friday, from 8:30 until 4:30pm.
If you are in an emergency situation or fear for your safety, call emergency services at 911, followed by HEC Montréal security services at 514 340-6611.
Check the school official policy on these matters for more details.
Family policy
HEC now has an official family policy, but the following guidelines reflect my own beliefs and commitments towards parent students2
- Babies are welcome in class as often as necessary for support feeding relationship.
- You are welcome to bring your child to class in order to cover unforeseeable gaps in childcare.
- If you come with babies or toddler, I ask that you sit close to the door so that, in case your little one needs special attention and is disrupting the learning of other students, you may step outside of class until their needs are met. Seats close to the door are reserved for parents attending class with their child.
References
Footnotes
There’s fairly widespread misunderstanding about what office hours actually are! Many students often think that they are the times I shouldn’t be disturbed, which is the exact opposite of what they’re for!↩︎
Shamelessly stolen/adapted from similar policy by Drs. Melissa Cheney, Guy Grossman and Rohan Alexander↩︎