Defining Diversity
I n chapter 1 we introduced the idea that an ensemble of predictors can be improved if the
ensemble members exhibit ‘diverse’ errors. In this chapter we further analyse the concept
of error “diversity”. We review existing explanations of why ensembles with diverse errors
perform well, for both regression and classification contexts. In the process of this we clarify
a subtle point, often overlooked, to do with quantifying classifier ensemble diversity and the
inherent non-ordinality of the predictor outputs. We proceed with a review of the literature
on attempts to create diversity in both forms, commenting on the structure of the field and
proposing a novel way to categorise different diversity creation techniques.