Krovetz and Croft (TOIS’92) reported extensive research on word sense
ambiguity using CACM and TIME test collections where the sense
disambiguation was done manually and they found that sense mismatches
occurred when documents were not relevant to queries. Their results
showed WS ambiguity causes surprisingly little degradation in IR and for
those corpus it seemed perfect WSD would yield only 2% improvement Voorhees built an automatic sense disambiguator based on WordNet and
tried it on a variety of standard test collections (SIGIR93) but got no
improvement in IR performance ... this was borne out by subsequent work
by others ... this is surprising and analysis has thrown up the evaluation of
wsd as an unknown quantity ... manual checking is too costly.