WeiYa's Work Yard

A dog, who fell into the ocean of statistics, tries to write down his ideas and notes to save himself.

Ensemble Learning

Posted on 0 Comments
Tags: Statistical Learning


build a prediction model by combining the strengths of a collection of simpler base models.

  • bagging and random forests (a committee of trees each cast a vote for the predicted class) are ensemble learning methods for classification
  • boosting (the committee of weak learners evolves over time, and the members cast a weighted vote.)
  • stacking (a novel approach to combining the strengths of a number of fitted models.)
  • Bayesian methods for nonparametric regression

two tasks

  • developing a population of base learners from the training data
  • combining them to form the composite predictor.

ECOC approach

Learning Ensemebles

consider functions of the form \(f(x) = \alpha_0 + \sum\limits_{T_k\in\cal T}\alpha_kT_k(x)\)

a hybrid approach which breaks this process down into two stages

  1. finite dicitionary $\cal T_L$
  2. family of functions $f_{\lambda}(x)$ built by fitting a lasso path

Published in categories Memo