The goal of training a classifier is not to learn an exact representation of the training data itself, but rather to build a statistical model of the process which generates the data.
This is crucial if the classifier is to exhibit good generalization,that is, to make good predictions for new inputs. This highlights the need to optimize the complexity of the model, for example, the number of adaptive weights in an artificial neural network. A network with many weights may have too much flexibility in relation to a particular
data set. On the other hand, a network model which is too simple may not be flexible enough to fit the data properly. This phenomenon is usually denoted as the bias-variance trade-off.