Assessing Recall in Information Retrieval
Peter Wallis
Information Technology Division,
DSTO,
Salisbury, Sth. Aust.
paw@itd.dsto.gov.au
Nicholas J. Redding
Information Technology Division,
DSTO,
Salisbury, Sth. Aust.
redding@itd.dsto.gov.au
Abstract
Information retrieval systems help people find books, articles, and other
chunks of coherent data which are relevant to some information need.
These systems perform well according to the accepted measures, but in
some situations, the accepted measures do not reflect the needs of the user.
One such situation is when the aim of the user is to find all relevant
material available.
Not only is it difficult to create an information retrieval system which
provides good recall, even if the system has retrieved everything
relevant, there is no known method of discovering that, indeed, the system
has been successful.
In this paper several previous proposals for assessing recall effectiveness
are described, and two methods of accurately measuring recall are discussed.
First, a method of creating a test-collection explicitly for recall is
provided which can be used to develop better retrieval engines.
Second, a statistical technique is outlined which provides the user with
reassurance that they have found everything relevant to their information need.
Conference Home Page