Bayesian Additive Regression Trees for Statistical Learning
Regression trees are flexible non-parametric models that are well suited to many modern statistical learning problems. Many such tree models have been proposed, from the simple single-tree model (e.g. Classification and Regression Trees — CART) to more complex tree ensembles (e.g. Random Forests). Their nonparametric formulation allows one to model datasets exhibiting complex non-linear relationships between predictors and the response. A recent innovation in the statistical literature is the development of a Bayesian analogue to these classical regression tree models. The benefit of the Bayesian approach is the ability to quantify uncertainties within a holistic Bayesian framework. We introduce the most popular variant, the Bayesian Additive Regression Trees (BART) model, and describe recent innovations to this framework. We conclude with some of the exciting research directions currently being explored.
Please note: This seminar has been cancelled.
Please note: This seminar has been cancelled.
Concentration of Maxima: Fundamental Limits of Exact Support Recovery in High Dimensions
We study the estimation of the support (set of non-zero components) of a sparse high-dimensional signal observed with additive and dependent noise. With the usual parameterization of the size of the support set and the signal magnitude, we characterize a phase-transition phenomenon akin to the Ingster’s signal detection boundary. We show that when the signal is above the so-called strong classification boundary, thresholding estimators achieve asymptotically perfect support recovery. This is so under arbitrary error dependence assumptions, provided that the marginal error distribution has rapidly varying tails. Conversely, under mild dependence conditions on the noise, we show that no thresholding estimators can achieve perfect support recovery if the signal is below the boundary. For log-concave error densities, the thresholding estimators are shown to be optimal and hence the strong classification boundary is universal, in this setting.
The proofs exploit a concentration of maxima phenomenon, known as relative stability. We obtain a complete characterization of the relative stability phenomenon for dependent Gaussian noise via Slepian, Sudakov-Fernique bounds and some Ramsey theory.
