Please use this identifier to cite or link to this item:

A general cost model for dimensionality reduction in high dimensional spaces

Authors Lian, Xiang
Chen, Lei
Issue Date 2007
Source International Conference on Data Engineering (ICDE) , 2007, p. 66-75
Summary Similarity search usually encounters a serious problem in the high dimensional space, known as the "curse of dimensionality". In order to speed up the retrieval efficiency, previous approaches usually reduce the dimensionality of the entire data set to a fixed lower value before building indexes (referred to as global dimensionality reduction (GDR)). More recent works focus on locally reducing the dimensionality of data to different values (called the local dimensionality reduction (LDR)). However so far little work has formally evaluated the effectiveness and efficiency of both GDR and LDR for range queries. Motivated by this, in this paper, we propose a general cost model for both GDR and LDR, in light of which we introduce a novel LDR method, PRANS. It can achieve high retrieval efficiency with the guarantee of optimality given by the formal model. Finally, a B(+) -tree index is constructed over the reduced partitions for fast similarity search. Extensive experiments validate the correctness of our cost model on both real and synthetic data sets, and demonstrate the efficiency and effectiveness of the proposed PRANS method.
ISSN 1084-4627
ISBN 978-1-4244-0802-3
Rights © 2007 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE. This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder.
Language English
Format Conference paper
Access View full-text via DOI
View full-text via Web of Science
View full-text via Scopus
Files in this item:
File Description Size Format
general.pdf 405975 B Adobe PDF