Most of the methods we have discussed manipulate input data. Dietterich and Bakiri [31]
manipulate the output targets with Error-Correcting Output Coding. Each output class in
the problem is represented by a binary string, chosen such that it is orthogonal (or as
close as possible) from the representation of the other classes. For a given input pattern, a
predictor is trained to reproduce the appropriate binary string. During testing, if the string
produced by the predictor does not exactly match the string representing one of the classes,
the Hamming distance is measured to each class, and the closest is chosen. Kong and
Dietterich [68] investigate why this technique works. They find that, like Bagging, ECOC
reduces the variance of the ensemble, but in addition can correct the bias component. An
important point to note for this result is that the 0-1 loss bias-variance decomposition
utilised assumes a Bayes rate of zero, i.e. zero noise.