Seven Numeric Properties of Effectiveness Metrics

Alistair Moffat
Department of Computing and Information Systems, The University of Melbourne, Victoria 3010, Australia.

Status

Proc. 9th Asian Information Retrieval Societies Conf., Singapore, December 2013, pages 1-12, LNCS volume 8281.

Abstract

Search effectiveness metrics quantify the relevance of the ranked document lists returned by retrieval systems. In this paper we characterize metrics according to seven numeric properties -- boundedness, monotonicity, convergence, top-weightedness, localization, completeness, and realizability. We demonstrate that these properties partition the commonly-used evaluation metrics, and hence provide a framework in which the relationships between effectiveness metrics can be better understood, including their relative merits for different applications.

Published paper

http://dx.doi.org/10.1007/978-3-642-45068-6_1