Outline
Instructor
- Dr. Léo Belzile
- 4.850, Côte-Sainte-Catherine
- leo.belzile@hec.ca
Course details
- Winter 2025
- Monday
- 15:30-18:30
- CSC, Labid Aljundi
Course content
Hands on introduction to Bayesian data analysis. The course will cover the formulation, evaluation and comparison of Bayesian models through examples.
Mathematical review. Basics of Markov Chain Monte Carlo methods and sampling algorithms, with a focus on off the shelf software (e.g., OpenBugs, Stan, INLA). Approximation methods. Hierarchical modelling, with a focus on latent Gaussian models.
Themes covered:
- Introduction to the Bayesian paradigm
- Formulation, comparison and evaluation of Bayesian models
- Sampling algorithms and Markov chain Monte Carlo methods
- Computational strategies for inference
- Hierarchical models
- Advanced topics
Target audience and prerequisites
The course is part of the PhD program in Administration offered by HEC Montréal jointly with McGill, Concordia and Université du Québec à Montréal (UQÀM). Other Québec students can register via BCI.
Course materials
I will provide slides and videos. In addition to those, there will be assigned readings from textbook and reference papers.
Textbooks
Course notes for the class can be found online
Additional references include Gelman et al. (2013), McElreath (2020) and Johnson et al. (2022).
- A. Gelman, J. Carlin, H. Stern, D. Dunson, A. Vehtari and D. Rubin (2013). Bayesian Data Analysis, 3rd edition, CRC press. doi: 10.1201/b16018
- R. McElreath (2020). Statistical Rethinking:
A Bayesian Course with Examples in R and Stan, 2nd edition, CRC Press. - A.A. Johnson, M.Q. Ott, and M. Dogucu (2022). Bayes Rules!
An Introduction to Applied Bayesian Modeling, 1st edition, CRC Press. Free available online.
Other references
There will occasionally be additional articles to read; links to these other resources will be included on the content page for that session.
Course content
Below is a tentative schedule.
Week 1: Tools of the trade
- Marginalization and conditioning
- Review of probability distributions
- Likelihood
- Monte Carlo integration
Week 2: Basics of Bayesian inference
- Key concepts: prior, posterior and interpretation
- Predictive distributions
- Marginal likelihood and numerical integration
- Credible intervals, loss functions and posterior summaries
- The beta binomial conjugate model
Week 3: Prior beliefs
- Conjugate priors
- Flat and vague priors.
- Priors for scale parameters
- Parameter elicitation and expert knowledge
- Penalized complexity prior
- Prior sensitivity analysis
Week 4: Markov chain Monte Carlo methods
- Monte Carlo methods
- Basics of Markov chains
- Metropolis–Hastings algorithm
Week 5: Markov chain Monte Carlo methods
- Basics of MCMC
- Metropolis-adjusted Langevin
- Gibbs sampling
- Data augmentation
Week 6: Computational strategies for MCMC
- Bayesian workflow
- Model diagnostics (WAIC, LOO-CV, etc.)
- Reparametrization
- Marginalization and joint updates
- Numerical approximations
- Optimal tuning of variance parameters
Week 7: Hamiltonian Monte Carlo
- Probabilistic programming
- Model comparison and Bayes factors
Week 8: Regression models
- Bayesian generalized linear model
- Shrinkage priors
- Random effects and pooling
Week 9: Hierarchical models
- Latent Gaussian models
- Multi-stage modelling
- Applications to time series and spatial data
Week 10: Deterministic approximations
- Asymptotics and Berstein-von Mises theorem
- Laplace approximation
- Integrated Laplace approximation
Week 11: Variational inference
- Gradient descent
- ELBO criterion and Kullback–Leibler divergence
- Variational Bayes
Week 12: Final review
- Expectation propagation
- Course recap
Evaluations
Your final grade will be based on weekly problem sets and check-ins, a midterm and a final examination. All evaluations are individual work.
The midterm and final are closed book exams. Students are forbidden to use generative artificial intelligence (AI) tools. Any reference should be adequately cited.
There will be weekly exercises pertaining to the class material, one of which must be handed in for credit. We will discuss the other exercises at the beginning of the next class.
The midterm will take place on Monday, March 3rd from 15:30–18:30. The final is cumulative and will take place on Tuesday, April 15th from 18:30–21:30.
Weekly check-in
Every week, after you finish working through the content, I want to hear about what you learned and what questions you still have. To facilitate this, and to encourage engagement with the course content, you’ll need to write a small feedback in ZoneCours. This should be ~150 words.
You should answer the following three questions each week:
- What was the most exciting thing you learned from the session? Why?
- What was the muddiest thing from the session this week? What are you still wondering about?
- Which activity did you find the most useful? What could have been skipped?
The weekly check-in is an occasion for you to ask for clarification, highlight areas or topics for which examples could be added, or list superfluous activities and content. I will grade these before class, answer individually through the feedback form or at the beginning of class.
I will grade these check-ins using a check system:
- 110%: Response shows phenomenal thought and engagement with the course content. I will not assign these often.
- 100%: Response is thoughtful, well-written, and shows engagement with the course content. This is the expected level of performance.
- 50%: Response is hastily composed, too short, and/or only cursorily engages with the course content. This grade signals that you need to improve next time. I will hopefully not assign these often.
Notice that is essentially a pass/fail or completion-based system. I’m not grading your writing ability, I’m not counting the exact number of words you’re writing, and I’m not looking for encyclopedic citations of every single reading to prove that you did indeed read everything. I’m looking for thoughtful engagement, that’s all. Do good work and you’ll get a ✔.
| Assignment | Points |
|---|---|
| Weekly check-in (10 × 0.5 pt) | 5 |
| Assignments | 30 |
| Midterm examination (30 pts) | 30 |
| Final examination (35 pts) | 35 |
| Total | 100 |
Student hours
Monday before class or by appointment. My office, 4.850, is located next to the southern elevators in Côte-Sainte-Catherine building.
Please watch this video:
Student hours are set times dedicated to all of you (most professors call these “office hours”; I don’t1). This means that I will be in my office waiting for you to come by if you want to talk to me in person (or remotely) with whatever questions you have. This is the best and easiest way to find me and the best chance for discussing class material and concerns.
Intellectual integrity
Please don’t cheat! The official policy lists the school rules regarding plagiarism and academic integrity.
Student services
Students with special needs should feel free to approach me so we can best discuss accommodations. Do check out HEC Montréal’s disabled students and psychological support services.
Harassment and sexual violence
The Center for Harassment Intervention (BIMH) is the unique access point for all members of the community subject to harassment or sexual violence. You can reach them at 514 343-7020 or by email at harcelement@hec.ca from Monday until Friday, from 8:30 until 4:30pm.
If you are in an emergency situation or fear for your safety, call emergency services at 911, followed by HEC Montréal security services at 514 340-6611.
Check the school official policy on these matters for more details.
Family policy
HEC now has an official family policy, but the following guidelines reflect my own beliefs and commitments towards parent students2
- Babies are welcome in class as often as necessary for support feeding relationship.
- You are welcome to bring your child to class in order to cover unforeseeable gaps in childcare.
- If you come with babies or toddler, I ask that you sit close to the door so that, in case your little one needs special attention and is disrupting the learning of other students, you may step outside of class until their needs are met. Seats close to the door are reserved for parents attending class with their child.
References
Footnotes
There’s fairly widespread misunderstanding about what office hours actually are! Many students often think that they are the times I shouldn’t be disturbed, which is the exact opposite of what they’re for!↩︎
Shamelessly stolen/adapted from similar policy by Drs. Melissa Cheney, Guy Grossman and Rohan Alexander↩︎