recursive divide-and-conquer manner and the compatibility of
Decision trees degrades because the output is limited to one
attribute. Trees created from the numeric datasets seems to be
more complex and also when the database is large the
complexity of the tree increases. In comparison with the
Random Forest algorithm the time complexity of Decision
trees increases exponentially with the tree height. Hence
shallow trees tend to have large number of leaves and high
error rates.
As the tree size increases, training error decreases. However,
as the tree size increases, testing error decreases at first since
we expect the test data to be similar to the training data, but at
a certain point, the training algorithm starts training to the
noise in the data, becoming less accurate on the testing data.
At this point we are no longer fitting the data and instead
fitting the noise in the data. This is called over fitting to the
data, in which the tree is fitted to spurious data. As the tree
grows in size, it will fit the training data perfectly and not be
of practical use for other data such as the testing set.