The bias can be characterised as a measure of how close, on average over several
different training sets, your estimator is to its target. The variance is a measure of how
‘stable’ the solution is: given slightly different training data, an estimator with high variance
will tend to produce wildly varying performance. Training an estimator for a long time tends
to decrease bias, but slowly increase variance; at some point there will be an optimal tradeoff that minimises the generalisation error. This is the bias-variance dilemma; a typical
training session is shown in figure