2.1.3 Bias and Variance
The most important theoretical tool in machine learning research is the bias-variance decomposition [45]. The original decomposition by Geman et al [45] applies to quadratic
loss error functions, and states that the generalisation error can be broken into separate
components each with their own interpretation. However as mentioned in section 2.1.1,
different tasks may require different error measures, not necessarily using quadratic loss.
Several authors have proposed decompositions for 0-1 loss [14, 42, 68, 67], each with their
own shortcomings and assumptions. Most recently Domingos [33] and James [58] provide
equivalent, unified definitions for any symmetric3 loss function. This thesis is only concerned with the quadratic loss decomposition, though in section 3.1.2 and the Conclusions
chapter we do discuss certain possible extensions using the 0-1 loss function.
The bias-variance decomposition for quadratic loss states that the generalisation error of
an estimator can be broken down into two components: bias and variance. These two usually
work in opposition to each other: attempts to reduce the bias component will cause an
increase in variance, and vice versa. Techniques in the machine learning literature are often
evaluated on how well they can optimize the trade-off between these two components
2.1.3 Bias and VarianceThe most important theoretical tool in machine learning research is the bias-variance decomposition [45]. The original decomposition by Geman et al [45] applies to quadraticloss error functions, and states that the generalisation error can be broken into separatecomponents each with their own interpretation. However as mentioned in section 2.1.1,different tasks may require different error measures, not necessarily using quadratic loss.Several authors have proposed decompositions for 0-1 loss [14, 42, 68, 67], each with theirown shortcomings and assumptions. Most recently Domingos [33] and James [58] provideequivalent, unified definitions for any symmetric3 loss function. This thesis is only concerned with the quadratic loss decomposition, though in section 3.1.2 and the Conclusionschapter we do discuss certain possible extensions using the 0-1 loss function.The bias-variance decomposition for quadratic loss states that the generalisation error ofan estimator can be broken down into two components: bias and variance. These two usuallywork in opposition to each other: attempts to reduce the bias component will cause anincrease in variance, and vice versa. Techniques in the machine learning literature are oftenevaluated on how well they can optimize the trade-off between these two components
การแปล กรุณารอสักครู่..
