The simplest way to achieve this is to periodically reconstruct the index
from scratch. This is a good solution if the number of changes over time is
small and a delay in making new documents searchable is acceptable – and
if enough resources are available to construct a new index while the old one
is still available for querying.
If there is a requirement that new documents be included quickly, one soluAUXILIARY
INDEX tion is to maintain two indexes: a large main index and a small auxiliary index
that stores new documents. The auxiliary index is kept in memory. Searches
are run across both indexes and results merged. Deletions are stored in an invalidation
bit vector. We can then filter out deleted documents before returning
the search result. Documents are updated by deleting and reinserting
them