Exercises 9

Exercise 9.1

We consider the accuracy of the Gaussian approximation to the posterior of a Bernoulli likelihood $Y_i\sim \mathsf{binom}(1,\theta )$ with $y=\sum _{i=1}^n{y}_i=\lfloor 0.1n\rfloor$ successes out of $n$ trials, i.e., if 10% of the realizations are successes. To do so,

1. Obtain a closed-form expression maximum a posteriori and the hessian of the log posterior for a conjugate $\mathsf{beta}(a,b)$ prior.
2. Repeat this with $\vartheta =\log (\theta )-\log (1-\theta )$, using a change of variable.
3. Plug in the approximations with a $\theta \sim \mathsf{beta}(1/2,1/2)$ prior for $n\in \{25,50,100\}$ and plot the Gaussian approximation along with the true posterior. Is the approximation for $\vartheta$ better for small $n$? Discuss.
4. Compute the marginal likelihood and compare the approximation with the true value.

Exercise 9.2

Consider the Bernoulli sample of size $n$ with $y$ successes and a $\mathsf{beta}(1,1)$ conjugate prior. Compute the Laplace approximation to the posterior mean for samples of size $n=10,20,50,100$ and $y/n\in \{0,0.1,0.25,0.5\}$.