Bayesian Additive Regression Trees for Statistical Learning
Regression trees are flexible non-parametric models that are well suited to many modern statistical learning problems. Many such tree models have been proposed, from the simple single-tree model (e.g. Classification and Regression Trees — CART) to more complex tree ensembles (e.g. Random Forests). Their nonparametric formulation allows one to model datasets exhibiting complex non-linear relationships between predictors and the response. A recent innovation in the statistical literature is the development of a Bayesian analogue to these classical regression tree models. The benefit of the Bayesian approach is the ability to quantify uncertainties within a holistic Bayesian framework. We introduce the most popular variant, the Bayesian Additive Regression Trees (BART) model, and describe recent innovations to this framework. We conclude with some of the exciting research directions currently being explored.