What does it mean to "Measure Performance"?
Department of Computer Science and Software Engineering,
The University of Melbourne,
Victoria 3010, Australia.
Department of Computer Science,
RMIT, GPO Box 2476V,
Melbourne 3001, Australia.
Proc. Fifth International Conf. on Web
Informations Systems, Brisbane, Australia, November 2004, pages 1-12.
The purpose of much computer science research is to invent
algorithms, and generate evidence to convince others that the new
methods are worthwhile.
All too often, however, the work has no impact, because the
supporting evidence does not meet basic standards of rigor or
Here the notion of "experiment" is explored, with reference to our
current investigation into distributed text query processing on a
cluster of computers.
We describe some of the issues that we encountered, and lessons that
can be applied by researchers designing experiments in other areas of