A Compression-Based Digital Library


Alistair Moffat
Department of Computer Science, The University of Melbourne, Parkville 3052, Australia.

Ian H. Witten
Department of Computer Science, Waikato University, New Zealand.


Status

DESIDOC Bulletin of Information Technology, 17(6):31-41, November 1997.

Abstract

The explosive growth in digital information places an ever-increasing burden on the library mechanisms used to store and access it. A wide range of technologies must be developed to manage this flood of data. For example, we need ways of using storage effectively and economically, mechanisms for indexing, and methods for fast content-based querying. In this article we consider these three issues from an implementation point of view, showing how appropriate data compression techniques can be harnessed to provide improved solutions in all three areas. To set a context for the development, we discuss the New Zealand Digital Library initiative, a Web-based tool that provides many different services, the largest of which indexes in excess of 40,000 technical documents drawn from sources around the world.