# Ensemble Learning

##### Posted on May 17, 20170 Comments
Tags: Statistical Learning

## idea

build a prediction model by combining the strengths of a collection of simpler base models.

• bagging and random forests (a committee of trees each cast a vote for the predicted class) are ensemble learning methods for classification
• boosting (the committee of weak learners evolves over time, and the members cast a weighted vote.)
• stacking (a novel approach to combining the strengths of a number of fitted models.)
• Bayesian methods for nonparametric regression

• developing a population of base learners from the training data
• combining them to form the composite predictor.

## Learning Ensemebles

consider functions of the form $f(x) = \alpha_0 + \sum\limits_{T_k\in\cal T}\alpha_kT_k(x)$

#### a hybrid approach which breaks this process down into two stages

1. finite dicitionary $\cal T_L$
2. family of functions $f_{\lambda}(x)$ built by fitting a lasso path

Published in categories Memo