In a Classification Context
We know understand that in a regression context, we can rigorously define why and how
differences between individual predictor outputs contribute toward overall ensemble accuracy. In a classification context, there is no such neat theory. There is a subtle point here,
often overlooked. Difficulties in quantifying classification error diversity is not intrinsic to
ensembles tackling classification problems. It is possible to reformulate any classification
problem as a regression one by choosing to approximate the class posterior probabilities;
this allows the theory we have already discussed to apply, and work is progressing in this
area, notably1 Tumer and Ghosh [140] and Roli and Fumera [44, 113]. For the regression
context discussed in the previous section, the question can be clearly phrased as “how can
we quantify diversity when our predictors output real-valued numbers and are combined by
a convex combination?”. For the case that Tumer and Ghosh [140] study, the question is
the same, just that the “real-valued” numbers are probabilities. A much harder question