2.1.3 Bias and Variance
The most important theoretical tool in machine learning research is the bias-variance decomposition
[45]. The original decomposition by Geman et al [45] applies to quadratic
loss error functions, and states that the generalisation error can be broken into separate
components each with their own interpretation. However as mentioned in section 2.1.1,
different tasks may require different error measures, not necessarily using quadratic loss.
Several authors have proposed decompositions for 0-1 loss [14, 42, 68, 67], each with their
own shortcomings and assumptions. Most recently Domingos [33] and James [58] provide
equivalent, unified definitions for any symmetric3
loss function. This thesis is only concerned
with the quadratic loss decomposition, though in section 3.1.2 and the Conclusions
chapter we do discuss certain possible extensions using the 0-1 loss function.