Lecture 13: Ridge, Lasso and Mixtures in Bayesian Regression

Readings

• A First Course in Bayesian Statistical Methods by Peter D. Hoff

  • Section 9.1: The linear regression model (review)
  • Section 9.2: Bayesian estimation for a regression model

• Bayesian Data Analysis (Third Edition) by Andrew Gelman, John Carlin, Hal Stern, David Dunson, Aki Vehtari, and Donald Rubin

  • Section 14.5: Ssembling the Matrix of Explanatory Variables
  • Section 14.6: Regularization and Dimension Reduction

• The Bayesian Choice (Second Edition) by Christian Robert

Papers:

• Park, T. & Casella, G. (2008) The Bayesian Lasso, Journal of the American Statistical Association, 103:681-686
• Hans, C. (2009) Model uncertainty and variable selection in Bayesian lasso regression, Statistics and Computing 20:221–229
• Hans, C. (2010) Bayesian Lasso Regression, Biometrika, 96: 835-845
• Carvalho, C. Polson, N. and Scott, J (2010) The Horseshoe Estimator for Sparse Signals, Biometrika, 97:465-480
• Armagan, A. Dunson, D, and Lee, J. (2013) Generalized double Pareto shrinkage, Statistica Sinica 23:119-143 arXiv: 1104.0861v4
• Fan, J. and Li (2001) Variable Selection via Nonconcave Penalized Likelihood and Its Oracle Properties,[Journal of the American Statistical Association, 96:1348-1360] (https://www.jstor.org/stable/3085904)
• George, E.I. Liang, F., Xu, X., (2012) From Minimax Shrinkage Estimation to Minimax Shrinkage Prediction, Statistical Science, 27:82-94