Assignment 2

To be handed for credit by November 11th at 23:59 the latest.

Problem 2

Let \(Y_{ij1}\) (\(Y_{ij2}\)) denote the score of the home (respectively visitor) team for a soccer match opposing teams \(i\) and \(j\). Maher (1982) suggested modelling the scores as \[\begin{align*} Y_{ij1} \mid \delta, \alpha_i, \beta_j &\sim \mathsf{Poisson}\{\exp(\delta + \alpha_i +\beta_j)\}, \\ Y_{ij2} \mid \alpha_j, \beta_i &\sim \mathsf{Poisson}\{\exp(\alpha_j+\beta_i)\}, \end{align*}\] where

  • \(\alpha_i\) represent the offensive strength of the team,
  • \(\beta_j\) the defensive strength of team \(j\) and
  • \(\delta\) is the common home advantage.

The scores in a match and between matches are assumed to be conditionally independent of one another given \(\boldsymbol{\alpha}, \boldsymbol{\beta}, \delta\). To complete the specification of this model, we set \(\alpha_i \sim \mathsf{Gauss}(0, \sigma^2_\alpha)\) and \(\beta_i \sim\mathsf{Gauss}(0, \sigma^2_\beta)\) for \(i=1, \ldots, n\) and \(\delta \sim \mathsf{Gauss}(0, 2)\).

The data set efl contains the results of football (soccer) matches for the 2015 season of the English Premier Ligue (EPL) and contains the following variables

  • score: number of goals of team during a match
  • team: categorical variable giving the name of the team which scored the goals
  • opponent: categorical variable giving the name of the adversary
  • home: binary variable, 1 if team is playing at home, 0 otherwise.
  1. Specify penalized complexity priors (Simpson et al., 2017) for the scales \(\sigma_\alpha\) and \(\sigma_\beta\) such that they have a probability of 10% of exceeding 1.
  2. D’Angelo & Canale (2023) consider efficient proposals for Metropolis-Hastings step for Poisson regression models with Gaussian priors on the coefficients, building on a data augmentation scheme of Polson et al. (2013). Implement their MCMC algorithm [section 2] with their proposed tuning parameter scheme and check the correctness of your sampler.

This section describes the corrected algorithm of D’Angelo & Canale (2023), whose original version is riddled with typos (some also remain after correction).

The paper proposes a Gaussian proposal distribution when the likelihood \[\begin{align*} Y_i \mid \boldsymbol{\beta} &\stackrel{\text{ind}}{\sim} \mathsf{Poisson}\{\exp(\mathbf{X}_i\boldsymbol{\beta})\}\\ \boldsymbol{\beta} &\sim \mathsf{Gauss}_p(\mathbf{b}, \mathbf{B}). \end{align*}\] In the problem set, \(\mathbf{B}\) is a diagonal matrix and \(\mathbf{b}=\mathbf{0}_p.\)

The transition kernel proposal density \(q\) for \(\boldsymbol{\beta}\) is a Gaussian. Let \(\lambda_i(\boldsymbol{\beta}) = \exp(\mathbf{x}^\top_i\boldsymbol{\beta})\) denote the Poisson mean with predictor row vector \(\mathbf{x}_i\). The proposals below are a function of tuning parameters \(r_i\) \((i=1, \ldots, n)\) which are themselves deterministic functions of a single scalar tuning parameter \(d\). The transition kernel is not symmetric in it’s arguments, so must be evaluated at both the proposal vector \(\boldsymbol{\beta}^{\star}\) and current value \(\boldsymbol{\beta}^{(t-1)}\):

Step 1: For given \(d\) and \(\lambda_i =\exp(\mathbf{x}^\top_i\boldsymbol{\beta})\), estimate the tuning parameters \(\boldsymbol{r}\) as \[ r_i = \frac{\lambda_i\{\lambda_i + \log(d+1)\}}{\lambda_i + \log(d+1) + \lambda_i \mathrm{W}_0\left[\frac{-(d+1)^{-1/\lambda_i}}{e^1\lambda_i}\{\lambda_i + \log(d+1)\}\right]}. \] Step 2: Compute \[\omega_i=\frac{r_i+y_i}{2\{\log(\lambda_i) - \log(r_i)\}} \frac{\lambda_i - r_i}{\lambda_i+r_i}\] Step 3: Compute the transition kernel variance \(\boldsymbol{\Sigma}_e = \left( \mathbf{X}^\top \mathrm{diag}(\boldsymbol{\omega})\mathbf{X} + \mathbf{B}^{-1}\right)^{-1}.\)

Step 4: Compute the proposal mean \(\boldsymbol{\mu}_e\) \[\begin{align*} \boldsymbol{\mu}_e & = \boldsymbol{\Sigma}_e \left(\mathbf{X}^\top\boldsymbol{\kappa} + \mathbf{B}^{-1}\mathbf{b}\right)\\ \end{align*}\] where \(\boldsymbol{\kappa} = (\kappa_1, \ldots, \kappa_n)^\top\) is an \(n\)-vector with \(i\)th element \(\kappa_i = \omega_i \log(r_i) + (y_i-r_i)/2.\)

To check the correctness, I encourage you to simulate fake data from a Poisson hierarchical model with fixed \(\mathbf{b}\) and \(\mathbf{B}\) and run the MCMC sampler on these.

  1. Simulate posterior draws for this model applied to the EFL data. Run four Markov chains for 10K iterations.
  2. Plot the chains (including the burn in period) for a handful of parameters. How many iterations does it take for the chains to stabilize, if at all?
  3. Using the output of your Markov chains, estimate
    • the posterior mean
    • adjusted Monte Carlo standard errors for the posterior means,
    • the posterior quartiles (25%, 50% and 75%),
    • the \(\widehat{R}\) factors,
    • the effective sample size for each parameter.
  4. Answer the following questions:
    • Using the posterior distribution, give the expected number of goals for a match between Manchester United (at home) against Liverpool.
    • Report and interpret the estimated posterior mean home advantage \(\widehat{\delta}\).
    • Calculate the probability that the home advantage \(\delta\) is positive?
  5. One measure of adequacy for the model is obtained from leave-one-out cross-validation predictive density. Denote the vector of parameters by \(\boldsymbol{\theta} = (\boldsymbol{\alpha},\boldsymbol{\beta}, \delta, \sigma_\alpha, \sigma_\beta)\) and \(\boldsymbol{y}_{-i}\) the vector obtained by removing the \(i\)th observation \(y_i\) from the likelihood. Let \(f(y_i; \boldsymbol{\theta})\) denote the marginal density/mass function of sample \(y_i\). We consider the partial predictive density
    \[\begin{align*} f(y_i \mid \boldsymbol{y}_{-i})=\int_{\boldsymbol{\Theta}} f(y_i \mid \boldsymbol{\theta}) p(\boldsymbol{\theta} \mid \boldsymbol{y}_{-i}) \mathrm{d} \boldsymbol{\theta} \end{align*}\] Integrals with respect to the partial posterior \(p(\boldsymbol{\theta} \mid \boldsymbol{y}_{-i})\) can be approximated by taking instead draws from the full posterior \(p(\boldsymbol{\theta} \mid \boldsymbol{y})\) with importance weight \(w_i = 1/f(y_i; \boldsymbol{\theta})\) (Gelfand & Dey, 1994). Implement manually this importance sampling estimator for your model.1 Next compute the average log pointwise predictive density as a measure of goodness of fit.
  6. Maher also suggested more complex models, including one in which the offensive and defensive strength of each team changed depending on whether they were at home or visiting another team, i.e. \[\begin{align} Y_{ij1} \sim \mathsf{Poisson}\{\exp(\alpha_i +\beta_j + \delta)\}, Y_{ij2} \sim \mathsf{Poisson}\{\exp(\gamma_j+\omega_i)\}, \end{align}\] Fit the model with \(\gamma_i \sim \mathsf{Gauss}(0, \sigma^2_\gamma)\) and \(\omega_i \sim \mathsf{Gauss}(0, \sigma^2_\omega)\) using similar shrinkage priors for \(\sigma^2_\gamma\) and \(\sigma^2_\omega\). Does the second model fit significantly better than than the first? Compare the models using WAIC and the average log pointwise predictive density estimates.

References

D’Angelo, L., & Canale, A. (2023). Efficient posterior sampling for Bayesian Poisson regression. Journal of Computational and Graphical Statistics, 32(3), 917–926. https://doi.org/10.1080/10618600.2022.2123337
Gelfand, A. E., & Dey, D. K. (1994). Bayesian model choice: Asymptotics and exact calculations. Journal of the Royal Statistical Society: Series B (Methodological), 56(3), 501–514. https://doi.org/10.1111/j.2517-6161.1994.tb01996.x
Maher, M. J. (1982). Modelling association football scores. Statistica Neerlandica, 36(3), 109–118. https://doi.org/10.1111/j.1467-9574.1982.tb00782.x
Polson, N. G., Scott, J. G., & Windle, J. (2013). Bayesian inference for logistic models using pólya–gamma latent variables. Journal of the American Statistical Association, 108(504), 1339–1349. https://doi.org/10.1080/01621459.2013.829001
Simpson, D., Rue, H., Riebler, A., Martins, T. G., & Sørbye, S. H. (2017). Penalising model component complexity: A principled, practical approach to constructing priors. Statistical Science, 32(1), 1–28. https://doi.org/10.1214/16-sts576

Footnotes

  1. You should remove the results of a match \(i\) rather than a single score.↩︎