Assessing Recall in Information Retrieval

Peter Wallis
Information Technology Division, DSTO, Salisbury, Sth. Aust.

Nicholas J. Redding
Information Technology Division, DSTO, Salisbury, Sth. Aust.


Information retrieval systems help people find books, articles, and other chunks of coherent data which are relevant to some information need. These systems perform well according to the accepted measures, but in some situations, the accepted measures do not reflect the needs of the user. One such situation is when the aim of the user is to find all relevant material available. Not only is it difficult to create an information retrieval system which provides good recall, even if the system has retrieved everything relevant, there is no known method of discovering that, indeed, the system has been successful. In this paper several previous proposals for assessing recall effectiveness are described, and two methods of accurately measuring recall are discussed. First, a method of creating a test-collection explicitly for recall is provided which can be used to develop better retrieval engines. Second, a statistical technique is outlined which provides the user with reassurance that they have found everything relevant to their information need.
Conference Home Page