The most commonly used measure for model performance is its Accuracy de-
fined as the ratio between the instances that have been correctly classified (as be-
longing or not to the given class) and the total number of instances: Accuracy =
(T P + T N)/(T P + T N + FP + FN). However, accuracy might be misleading in
many cases. Imagine a 2-class problem in which there are 99,900 samples of class
A and 100 of class B. If a classifier simply predicts everything to be of class A,
the computed accuracy would be of 99.9% but the model performance is question-
able because it will never detect any class B examples. One way to improve this
evaluation is to define the cost matrix where we declare the cost of misclassifying
class B examples as being of class A. In real world applications different types of
errors may indeed have very different costs. For example, if the 100 samples above
correspond to defective airplane parts in an assembly line, incorrectly rejecting a
non-defective part (one of the 99,900 samples) has a negligible cost compared to
the cost of mistakenly classifying a defective part as a good part.