ECOC

Posted on Aug 18, 2018

The note is for Dietterich, T. and Bakiri, G. (1995). Solving multiclass learning problems via error-correcting output codes, Journal of Artificial Intelligence Research 2: 263–286..

Abstract

Approaches to multiclass learning problems

1. direct application of multiclass algorithms, such as C4.5 and CART
2. application of binary concept learning algorithms to learn individual binary functions for each of the $k$ classes
3. application of binary concept learning algorithms with distributed output representations.

New techinique

Error-correcting codes are employed as a distributed output representation

It is robust with respect to

1. changes in the size of the training sample
2. assignment of distributed representations to particular pruning 3, application of overfitting avoidance techniques such as decision-tree pruning.

And it can provide reliable class probability estimates.

Introduction

Two class

1. decision-tree methods, such as C4.5 and CART.
2. artificial neural network algorithms, such as the perceptron algorithm and the error BP algorithm.

Multiclass

1. direct multiclass approach: generalize decision-tree algorithms
2. one-per-class approach: learn one binary function for each class with connectionist algorithms. ($f_i,i=1,2,\ldots,k$)
3. distributed output code.