INST: An Adaptive Metric for Information Retrieval Evaluation

Alistair Moffat
Department of Computing and Information Systems, The University of Melbourne, Victoria 3010, Australia.

Peter Bailey
Microsoft Research, Canberra, Australia.

Falk Scholer
School of Computer Science and Information Technology, RMIT University, Victoria 3001, Australia.

Paul Thomas
CSIRO, Canberra Australia

Status

Proc. 20th Australasian Document Computing Symp., Parramatta, December 2015, pages 5:1-5:4.

Abstract

A large number of metrics have been proposed to measure the effectiveness of information retrieval systems. Here we provide a detailed explanation of one recent proposal, INST, articulate the various properties that it embodies, and describe a number of pragmatic issues that need to be taken in to account when writing an implementation. The result is a specification for a program inst_eval for use in TREC-style IR experimentation.

Full text

http://dx.doi.org/10.1145/2838931.2838938 .

Software

An implementation based on this paper has been prepared by Bevan Koopman, https://github.com/ielab/inst_eval.