CHAPTER 3. DEFINING DIVERSITY 46
context, the portion of the ensemble error corresponding to diversity is the covariance term.
It can be seen that this does not “correlate well” with the mean squared error, since it is
intrinsically part of the ensemble error. Liu [85] conducted studies of the bias, variance and
covariance of an ensemble, illustrating the nature of this three-way trade-off. This work
showed that diversity (the covariance term) can be pushed too far, causing the other two
components to rise. We emphasize that most probably the same principle will one day be
shown for the ensemble majority vote accuracy. It can easily be shown in a hypothetical
scenario, that maximum classification diversity can in be harmful if individual accuracy is