We found no significant differences among models M3 and
M4 that means that these two models achieved almost the
same test error rate in Table III and almost the same Type II
and Type I error in Tables IV and V, for different training
size. It is obvious from Table III, that test error rate decreases
proportionally to the learning sample size, this improvement
can be suitable to the estimate of models’ parameters which
become more precise with the increase of the training data
size. Tables IV and V, shows that Type II error and Type I
proportionally decrease to the design sample size, these results
prove the importance of the population size in classification.
As shown in Table III, the test error rate of the two previous
models achieved 0.308 which is the lowest rate of misclassified
instances, according to this first criterion these models are the
two best classification models. For the remaining models, we
remark that models M5 and M6 also achieved good results,
followed by model M2, the left behind two models generate
the most raised test error rate, specially model M1 which
appears the worst one.
Test error rate, however measured, is only one aspect
of performance, this criterion may not be the most precise
one, further misclassification rate can be another aspect of
performance, so each model is evaluated by assessing Type I
and II error rate. We remind that the cut-off threshold used
in this case study is 0.5 for this threshold, all the applicants
whose estimated probability of non-reliability P(Y = 0) is
less than 0.5 are assessed as non-reliable applicants, otherwise
they are classified as reliable. In Table IV model M1 and M7
achieved 0.185 and 0.203 error rate, which are the lowest Type
II error rate, in other hand models M5 and M6, followed by
M2 have the most raised rate, this kind of error arise when
a reliable applicant is predicted as non-reliable. Models M5
and M6 are less efficient in the reliable applicants prediction.
Table V summarize Type I error for the seven models. A
Type I error means taking a non-reliable client and predicting