Consider a set of documents with two equal sized classes that are split into two clusters.
If there is one cluster that contains few points and one cluster that contains
almost all the points, then the entropy of the resulting solution is very
nearly the entropy the larger cluster, which is just – ½ log2 ½ - ½ log2 ½ =
1. If the two clusters are equal size and the clustering algorithm manages
to produce somewhat pure clusters, i.e., both clusters are more than 50%
pure, then the entropy of each cluster is less than 1. The accompanying
table shows the entropy for some values.