Focused crawling attracted much att

Focused crawling attracted much attention in the early days of web search.
Menczer and Belew (1998) and Chakrabarti et al.
(1999) wrote two of the most influential papers.
Menczer and Belew (1998) envision a focused crawler made of autonomous software agents,
principally for a single user.
The user enters a list of both URLs and keywords.
The agent then attempts to find web pages that would be useful to the user,
and the user can rate those pages to give feedback to the system.
Chakrabarti et al. (1999) focus on crawling for specialized topical indexes.
Their crawler uses a classifier to determine the topicality of crawled pages, as well as a distiller, which judges the quality of a page as a source of links to other topical pages.
They evaluate their system against a traditional, unfocused crawler to show that an unfocused crawler seeded with topical links is not sufficient to achieve a topical crawl.
The broad link structure of the Web causes the unfocused crawler to quickly drift to other topics, while the focused crawler successfully stays on topic.