Document retrieval

Document Retrieval Systems
– Document retrieval systems match text records against user queries
– Consist of a database of documents, a classification algorithm, and a user interface
– Main tasks are finding relevant documents and evaluating matching results
– Internet search engines are classical applications of document retrieval
– Range from simple Boolean systems to systems using statistical or natural language processing techniques

Indexing Schemata
– Two main classes of indexing schemata: form-based and content-based
– Form-based indexing addresses the exact syntactic properties of a text
– Content-based approach exploits semantic connections between documents and queries
– Most content-based systems use an inverted index algorithm
– Signature file is a technique that creates a quick filter for matching documents

Form-based Indexing
– Addresses the exact syntactic properties of a text
– Text is generally unstructured and not necessarily in a natural language
– Used for processing large sets of chemical representations in molecular biology
– Suffix tree algorithm is an example of form-based indexing

Content-based Indexing
– Exploits semantic connections between documents and queries
– Most content-based systems use an inverted index algorithm
– Signature file is a technique for creating a quick filter
– Can beat inverted files in certain environments with proper parameters
– Involves creating a hash coded version of each file for matching

Example: PubMed
– PubMed form interface features related articles search
– Comparison of words from document title, abstract, and MeSH terms
– Uses a word-weighted algorithm for relevance ranking
– PubMed is a widely used document retrieval system
– Provides access to a vast collection of biomedical literature