Architectures for Learning a Set of Predictors
In this section we review the two main architectures for learning a set of predictors, ensembles and Mixtures of Experts. These two are very often seen as opposites, one “modular”
and one “non-modular”. The supposed boundary here can easily be blurred as we illustrate
a third technique which is claimed to blend smoothly between the two; we discuss this claim
and the relationship between all three systems.