Please use this identifier to cite or link to this item: http://hdl.handle.net/1783.1/23

Document ranking on weight-partitioned signature files

Authors Lee, Dik Lun
Ren, Liming
Issue Date 1994-12-16
Summary A signature file organization, called the weight-partitioned signature file, for supporting document ranking is proposed. It employs multiple signature files, each of which corresponds to one term frequency, to represent terms with different term frequencies. Words with the same term frequency in a document are grouped together and hashed into the signature file corresponding to that term frequency. This eliminates the need to explicitly record the term frequency for each word. We investigate the effect of false drops on retrieval effectiveness if they are not eliminated in the search process. We have shown that false drops introduce insignificant degradation on precision and recall when the false drop probability is below a certain threshold. This is an important result since false drop elimination could become the bottleneck in systems using fast signature file search techniques. We perform an analytical study on the performance of the weight-partitioned signature file under different search strategies and configurations. An optimal formula is obtained to determine for a fixed total storage overhead the storage to be allocated to each partition in order to minimize the effect of false drops on document ranks. Experiments were performed using two document collections to support the analytical results.
Language English
Format Technical report
Access
Files in this item:
File Description Size Format
tr94-39.pdf 296.06 kB Adobe PDF