Several security evaluation methods have been proposed
in the past. For example, the Orange Book [12] and the
Common Criteria for Information Technology Security
Evaluation [8] define a set of generic rules that allow
developers to specify the security attributes of their products,
and evaluators to verify if products actually meet their
claims. Another example is the red team strategy [12], which
consists of a group of experts trying to hack a given
computer system to evaluate security. To the best of our
knowledge, none of these security evaluation methods is
oriented towards security comparison, and they are too
complex to be used to compare software packages in real
environments, where the administrators have limited
resources and security knowledge.