ABSTRACT
We show that incorporating user behavior data can significantly
improve ordering of top results in real web search setting. We
examine alternatives for incorporating feedback into the ranking
process and explore the contributions of user feedback compared
to other common web search features. We report results of a large
scale evaluation over 3,000 queries and 12 million user
interactions with a popular web search engine. We show that
incorporating implicit feedback can augment other features,
improving the accuracy of a competitive web search ranking
algorithms by as much as 31% relative to the original
performance.