The entropy value is 0 if all instances are in a single class, and reaches itsmaximum
when each class has the same number of instances. The entropy measure derives
from information theory.
When a set S is split into multiple sets Si , i = 1, 2, . . . , r , we can measure the
purity of the resultant set of sets as: