We have observed that simple statistical discriminant or clusterization
methods in many cases give results that are comparable or
better than those found by neural networks. A review of different
approaches to classification and comparison of performance of 20
methods on 22 real world datasets has been done within the StatLog
European Community project [3]. The algorithms that appeared
most frequently as the top five were all of statistical nature,
including four discriminant approaches: linear (LDA), logistic
(LogDA), quadratic discriminant (QDA) and a more involved
DIPOL92 method that uses hyperplanes to discriminate between
clusters. The last of the top five method, ALLOC80, is based on
clusterization/density estimation techniques. It is also worth noting
that the simplest version of the nearest neighbor method, using a single
neighbor, achieved best results in 4 cases, and close to the best
in another 3 cases. Our own results on several medical datasets [4]
showed that logical rules that discriminate using hyperplanes perpendicular
to the axes of the input variables are sometimes more
accurate than any other method of classification. Among the top 5
algorithms MLPs trained with the backpropagation algorithm appear
only once at the third position and 3 times at the fifth position. This
clearly shows that in most cases MLPs did not find solutions as good
as those found by statistical discriminant function methods.