Although this document score sums term weights for just the matching query
terms, with relevance feedback the query can be expanded to include other important
terms from the relevant set. Note that if we have no relevance information,
we can set r and R to 0, which would give a pi value of 0.5, and would produce
the idf-like term weight discussed before.
So how good is this document score when used for ranking? Not very good,
it turns out. Although it does provide a method of incorporating relevance information,
in most cases we don’t have this information and instead would be
using term weights that are similar to idf weights. The absence of a tf component
makes a significant difference to the effectiveness of the ranking, and most
effectiveness measures will drop by about 50% if the ranking ignores this information.
This means, for example, that we might see 50% fewer relevant documents
in the top ranks if we used the binary independence model ranking instead of the
best tf.idf ranking.
It turns out, however, that the binary independence model is the basis for one
of the most effective and popular ranking algorithms, known as BM25.