A further challenge in adapting scalability techniques from data mining is the extreme sparsity of ratings in recommender systems.
The value of a recommender system lies in the fact that most customers have not evaluated most of the products. A typical bookstore
customer may rate thirty books and seek recommendations from over a million books in print. While thirty books may represent a
substantial opinion, will there be enough other customers whose reading histories overlap with this .003% of the catalog?
Researchers are exploring a variety of techniques to bridge the gaps caused by sparsity. Examples include explicitly supporting
transitivity in neighborhood formation and using dimensionality reduction to shrink the effective dimensionality of the product
space. Many classical dimensionality reduction techniques work poorly with extremely sparse data, so these techniques will have to
be modified to work for recommender systems.