SIGIR'98 papers: Aspect Windows, 3-D Visualizations, and Indirect Comparisons of Information Retrieval Systems

Aspect Windows, 3-D Visualizations, and Indirect Comparisons of Information Retrieval Systems


Russell Swan
Center for Intelligent Information Retrieval, Department of Computer Science, The University of Massachusetts, Amherst, MA 01003 USA

James Allan
Center for Intelligent Information Retrieval, Department of Computer Science, The University of Massachusetts, Amherst, MA 01003 USA


Abstract

We built two Information Retrieval systems that were targeted for the TREC-6 ``aspect oriented'' retrieval track. The systems were built to test the usefulness of different visualizations in an interactive IR setting---in particular, an ``aspect window'' for the chosen task, and a 3-D visualization of document inter-relationships. We studied 24 users of the system in order to investigate: whether the systems were more effective than a control system, whether experienced users outperformed novices, whether spatial reasoning ability was a good predictor of effective use of 3-D, and whether the systems could be compared indirectly via a control system. Our results show substantial differences in user performance are related to spatial reasoning ability and to a lesser degree other traits. We also obtained markedly different results from the direct and indirect comparisons.


SIGIR'98
24-28 August 1998
Melbourne, Australia.
sigir98@cs.mu.oz.au.