Exercise 4

Exercise 4.1

The Pareto distribution with shape \(\alpha>0\) and scale \(\tau>0\) has density \[ f(x; \alpha, \tau) = \alpha x^{-\alpha-1}\tau^\alpha \mathsf{I}(x > \tau). \] It can be used to model power laws in insurance and finance, or in demography. The uscitypopn data set in the hecbayes package contains the population size of cities above 200K inhabitants in the United States, from the 2020 census.

  1. Using improper priors, write the joint posterior for a simple random sample of size \(n\) and derive the conditional distributions \(p(\alpha \mid \boldsymbol{y}, \tau)\) and \(p(\tau \mid \alpha, \boldsymbol{y})\). Hint: the conditional density \(p(\alpha \mid \boldsymbol{y}, \tau)\) is that of a gamma; use the fact that \(m^\alpha=\exp\{\alpha\log(m)\}\) for \(m>0\).
  2. The mononomial distribution \(\mathsf{Mono}(a,b)\) has density \(p(x) \propto x^{a-1}\mathsf{I}(0 \leq x \leq b)\) for \(a, b > 0\). Find the normalizing constant for the distribution and obtain the quantile function to derive a random number generator.
  3. Implement Gibbs sampling for this problem for the uscitypopn data. Draw enough observations to obtain an effective sample size of at least 1000 observations. Calculate the accuracy of your estimates.

Exercise 4.2

Implement the Bayesian LASSO for the diabetes cancer surgery from package lars. Check Park & Casella (2008) for the details of the Gibbs sampling.

  1. Fit the model for a range of values of \(\lambda\) and produce parameter estimate paths to replicate Figure 2 of the paper.
  2. Check the effective sample size and comment on the mixing. Is it impacted by the tuning parameter?
  3. Implement the method of section 3.1 from Park & Casella (2008) by adding \(\lambda\) as a parameter.
  4. For three models with different values of \(\lambda\), compute the widely applicable information criterion (WAIC) and use it to assess predictive performance.

References

Park, T., & Casella, G. (2008). The Bayesian Lasso. Journal of the American Statistical Association, 103(482), 681–686. https://doi.org/10.1198/016214508000000337