Ensemble learning schemes are a new field in data mining. While current research
concentrates mainly on improving the performance of single learning
algorithms, an alternative is to combine learners with different biases. Stacking
is an ensemble learning scheme which tries to combine learners’ predictions or
confidences via another learning algorithm. However, the adoption of Stacking
into the data mining community is hampered by its large parameter space, consisting
mainly of other learning algorithms: (1) the set of learning algorithms
to combine, (2) the meta-learner responsible for the combining and (3) the type
of meta data to use: confidences or predictions. None of these parameters are
obvious choices. Furthermore, little is known about the relation between parameter
settings and performance of Stacking. By exploring all of Stacking’s
parameter settings and their interdependencies, we intend to make Stacking a
suitable choice for mainstream data mining applications. This chapter is based
on the paper (Seewald, 2002c).