Mixtures of Experts
The Mixtures of Experts architecture is a widely investigated paradigm for creating a combination of estimators [54, 55]. The principle underlying the architecture is that certain
estimators will be able to ‘specialise’ to particular parts of the input space. A Gating network receives the same inputs as the component estimators, but its outputs are used as
the combination weights. The Gating network is responsible for learning the appropriate
weighted combination of the specialised estimators (experts) for any given input. In this
way the input space is ‘carved-up’ between the experts, increasing and decreasing their
weights for particular patterns. Figure 2.5 illustrates the basic architecture.