Expectation propagation
Content
- Newton method and sequences of Gaussian approximations
- Expectation propagation (EP)
Learning objectives
At the end of the chapter, students should be able to
- implement expectation propagation in simple problems
- explain the differences between EP, Laplace and variational inference
- understand under which circumstances EP is unstable
Readings
Complementary readings
- Minka (2001)
Slides
View all slides in new window Download PDF of all slides
Code
Bishop, C. M. (2006). Pattern recognition and machine learning (information science and statistics). Springer-Verlag.
Cseke, B., & Heskes, T. (2011). Approximate marginals in latent Gaussian models. Journal of Machine Learning Research, 12(13), 417–454. http://jmlr.org/papers/v12/cseke11a.html
Dehaene, G., & Barthelmé, S. (2018). Expectation propagation in the large data limit. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 80(1), 199–217. https://doi.org/10.1111/rssb.12241
Minka, T. P. (2001). A family of algorithms for approximate Bayesian inference [PhD thesis, Massachusetts Institute of Technology]. http://hdl.handle.net/1721.1/86583