Ensemble Learning
Posted on 0 Comments
idea
build a prediction model by combining the strengths of a collection of simpler base models.
- bagging and random forests (a committee of trees each cast a vote for the predicted class) are ensemble learning methods for classification
- boosting (the committee of weak learners evolves over time, and the members cast a weighted vote.)
- stacking (a novel approach to combining the strengths of a number of fitted models.)
- Bayesian methods for nonparametric regression
two tasks
- developing a population of base learners from the training data
- combining them to form the composite predictor.
ECOC approach
Learning Ensemebles
consider functions of the form \(f(x) = \alpha_0 + \sum\limits_{T_k\in\cal T}\alpha_kT_k(x)\)
a hybrid approach which breaks this process down into two stages
- finite dicitionary $\cal T_L$
- family of functions $f_{\lambda}(x)$ built by fitting a lasso path