As illustrated by Figure 23.8, if the model is too simple in comparison with the
target function, then the learned function may not be capable of approximating the
target function, making it under-fit (Figure 23.8a). On the other hand, if the model
is too complex it may start trying to approximate irrelevant information (e.g. noise
that may be contained in the output values) which will cause the learned function
to over-fit the target function (Figure 23.8b). A possible solution to this is to have a
number of candidate models. The goal of model selection (MS) is thus to determine
the weights of the models in the ensemble, or in the case of a single model being
used, to select an appropriate one (Figure 23.8c):