Please use this identifier to cite or link to this item:

Orthogonal Sparse PCA and Covariance Estimation via Procrustes Reformulation

Authors Benidis, Konstantinos HKUST affiliated (currently or previously)
Sun, Ying HKUST affiliated (currently or previously)
Babu, Prabhu Sing HKUST affiliated (currently or previously)
Palomar, Daniel Perez View this author's profile
Issue Date 2016
Source IEEE Transactions on Signal Processing , v. PP, (99), 2016, article number 7558183
Summary The problem of estimating sparse eigenvectors of a symmetric matrix attracts a lot of attention in many applications, especially those with high dimensional data set. While classical eigenvectors can be obtained as the solution of a maximization problem, existing approaches formulate this problem by adding a penalty term into the objective function that encourages a sparse solution. However, the vast majority of the resulting methods achieve sparsity at the expense of sacrificing the orthogonality property. In this paper, we develop a new method to estimate dominant sparse eigenvectors without trading off their orthogonality. The problem is highly non-convex and hard to handle. We apply the minorization-maximization framework where we iteratively maximize a tight lower bound (surrogate function) of the objective function over the Stiefel manifold. The inner maximization problem turns out to be a rectangular Procrustes problem, which has a closed form solution. In addition, we propose a method to improve the covariance estimation problem when its underlying eigenvectors are known to be sparse. We use the eigenvalue decomposition of the covariance matrix to formulate an optimization problem where we impose sparsity on the corresponding eigenvectors. Numerical experiments show that the proposed eigenvector extraction algorithm outperforms existing algorithms in terms of support recovery and explained variance, while the covariance estimation algorithms improve significantly the sample covariance estimator. © 1991-2012 IEEE.
ISSN 1053-587X
Language English
Format Article
Access View full-text via DOI
View full-text via Scopus
View full-text via Web of Science