Relevance vs Marginal Relevance
A document can be redundant even if it is highly relevant
Duplicates
The same information from different sources
Marginal relevance is a better measure of utility for the user.
Using facts/entities as evaluation units more directly measures true relevance.
But harder to create evaluation set