# Ensemble Learning

##### Posted on 0 Comments

## idea

build a prediction model by combining the strengths of a collection of simpler base models.

- bagging and random forests (a committee of trees each cast a vote for the predicted class) are ensemble learning methods for classification
- boosting (the committee of weak learners evolves over time, and the members cast a weighted vote.)
- stacking (a novel approach to combining the strengths of a number of fitted models.)
- Bayesian methods for nonparametric regression

## two tasks

- developing a population of base learners from the training data
- combining them to form the composite predictor.

## ECOC approach

## Learning Ensemebles

consider functions of the form \(f(x) = \alpha_0 + \sum\limits_{T_k\in\cal T}\alpha_kT_k(x)\)

#### a hybrid approach which breaks this process down into two stages

- finite dicitionary $\cal T_L$
- family of functions $f_{\lambda}(x)$ built by fitting a lasso path