CHAPTER 1. INTRODUCTION 3
bers can be recognised and managed, leading to the correct decision being taken overall.
Several empirical results [66, 114, 115, 149] have supported the widespread view that for
best performance on a task, the individuals should exhibit “diverse errors”. The intuition
here is that with a committee of people making decisions, we would not want them to all
make the same bad judgements at the same time. However in order to implement these
ideas in a computer system, they need to be far more formalised. What exactly do we
mean by “diverse” errors? The answer to this question depends critically on whether our
predictor outputs are real-valued numbers, or class labels. A real-valued number will be
output if the problem is a regression, say for example predicting 83.74kg as the weight of
a person given their height and shoe-size. A class label will be output if the problem is a
classification, for example predicting Y ES as the guess of whether the person is over 80.0kg
or not. The former situation, regression, is reasonably well understood (see section 3.1.1),
however when “diversity” is referred to it is usually intended to imply the latter, classi-
fication error diversity. Intuitively, we know that the predictors (in this case, classifiers)
should make different errors, yet formalising how they should do this has proved a difficult
challenge. Though the predictors may exhibit very different errors, they may have sacrificed
individual accuracy in order to do so; this shows that training an ensemble is a balancing
act between error diversity and individual accuracy. This issue will be addressed in more
detail in chapter 3.