-
Improved Pan-Private Stream Density Estimation
Authors:
Vassilis Digalakis Jr,
George N. Karystinos,
Minos N. Garofalakis
Abstract:
Differential privacy is a rigorous definition for privacy that guarantees that any analysis performed on a sensitive dataset leaks no information about the individuals whose data are contained therein. In this work, we develop new differentially private algorithms to analyze streaming data. Specifically, we consider the problem of estimating the density of a stream of users (or, more generally, el…
▽ More
Differential privacy is a rigorous definition for privacy that guarantees that any analysis performed on a sensitive dataset leaks no information about the individuals whose data are contained therein. In this work, we develop new differentially private algorithms to analyze streaming data. Specifically, we consider the problem of estimating the density of a stream of users (or, more generally, elements), which expresses the fraction of all users that actually appear in the stream. We focus on one of the strongest privacy guarantees for the streaming model, namely user-level pan-privacy, which ensures that the privacy of any user is protected, even against an adversary that observes, on rare occasions, the internal state of the algorithm. Our proposed algorithms employ optimally all the allocated privacy budget, are specially tailored for the streaming model, and, hence, outperform both theoretically and experimentally the conventional sampling-based approach.
△ Less
Submitted 16 November, 2021;
originally announced November 2021.
-
Optimal Algorithms for $L_1$-subspace Signal Processing
Authors:
Panos P. Markopoulos,
George N. Karystinos,
Dimitris A. Pados
Abstract:
We describe ways to define and calculate $L_1$-norm signal subspaces which are less sensitive to outlying data than $L_2$-calculated subspaces. We start with the computation of the $L_1$ maximum-projection principal component of a data matrix containing $N$ signal samples of dimension $D$. We show that while the general problem is formally NP-hard in asymptotically large $N$, $D$, the case of engi…
▽ More
We describe ways to define and calculate $L_1$-norm signal subspaces which are less sensitive to outlying data than $L_2$-calculated subspaces. We start with the computation of the $L_1$ maximum-projection principal component of a data matrix containing $N$ signal samples of dimension $D$. We show that while the general problem is formally NP-hard in asymptotically large $N$, $D$, the case of engineering interest of fixed dimension $D$ and asymptotically large sample size $N$ is not. In particular, for the case where the sample size is less than the fixed dimension ($N<D$), we present in explicit form an optimal algorithm of computational cost $2^N$. For the case $N \geq D$, we present an optimal algorithm of complexity $\mathcal O(N^D)$. We generalize to multiple $L_1$-max-projection components and present an explicit optimal $L_1$ subspace calculation algorithm of complexity $\mathcal O(N^{DK-K+1})$ where $K$ is the desired number of $L_1$ principal components (subspace rank). We conclude with illustrations of $L_1$-subspace signal processing in the fields of data dimensionality reduction, direction-of-arrival estimation, and image conditioning/restoration.
△ Less
Submitted 27 May, 2014;
originally announced May 2014.
-
Fixed-rank Rayleigh Quotient Maximization by an $M$PSK Sequence
Authors:
Anastasios Kyrillidis,
George N. Karystinos
Abstract:
Certain optimization problems in communication systems, such as limited-feedback constant-envelope beamforming or noncoherent $M$-ary phase-shift keying ($M$PSK) sequence detection, result in the maximization of a fixed-rank positive semidefinite quadratic form over the $M$PSK alphabet. This form is a special case of the Rayleigh quotient of a matrix and, in general, its maximization by an $M$PSK…
▽ More
Certain optimization problems in communication systems, such as limited-feedback constant-envelope beamforming or noncoherent $M$-ary phase-shift keying ($M$PSK) sequence detection, result in the maximization of a fixed-rank positive semidefinite quadratic form over the $M$PSK alphabet. This form is a special case of the Rayleigh quotient of a matrix and, in general, its maximization by an $M$PSK sequence is $\mathcal{NP}$-hard. However, if the rank of the matrix is not a function of its size, then the optimal solution can be computed with polynomial complexity in the matrix size. In this work, we develop a new technique to efficiently solve this problem by utilizing auxiliary continuous-valued angles and partitioning the resulting continuous space of solutions into a polynomial-size set of regions, each of which corresponds to a distinct $M$PSK sequence. The sequence that maximizes the Rayleigh quotient is shown to belong to this polynomial-size set of sequences, thus efficiently reducing the size of the feasible set from exponential to polynomial. Based on this analysis, we also develop an algorithm that constructs this set in polynomial time and show that it is fully parallelizable, memory efficient, and rank scalable. The proposed algorithm compares favorably with other solvers for this problem that have appeared recently in the literature.
△ Less
Submitted 27 January, 2014;
originally announced January 2014.
-
The Sparse Principal Component of a Constant-rank Matrix
Authors:
Megasthenis Asteris,
Dimitris S. Papailiopoulos,
George N. Karystinos
Abstract:
The computation of the sparse principal component of a matrix is equivalent to the identification of its principal submatrix with the largest maximum eigenvalue. Finding this optimal submatrix is what renders the problem ${\mathcal{NP}}$-hard. In this work, we prove that, if the matrix is positive semidefinite and its rank is constant, then its sparse principal component is polynomially computable…
▽ More
The computation of the sparse principal component of a matrix is equivalent to the identification of its principal submatrix with the largest maximum eigenvalue. Finding this optimal submatrix is what renders the problem ${\mathcal{NP}}$-hard. In this work, we prove that, if the matrix is positive semidefinite and its rank is constant, then its sparse principal component is polynomially computable. Our proof utilizes the auxiliary unit vector technique that has been recently developed to identify problems that are polynomially solvable. Moreover, we use this technique to design an algorithm which, for any sparsity value, computes the sparse principal component with complexity ${\mathcal O}\left(N^{D+1}\right)$, where $N$ and $D$ are the matrix size and rank, respectively. Our algorithm is fully parallelizable and memory efficient.
△ Less
Submitted 20 December, 2013;
originally announced December 2013.
-
Sparse Principal Component of a Rank-deficient Matrix
Authors:
Megasthenis Asteris,
Dimitris S. Papailiopoulos,
George N. Karystinos
Abstract:
We consider the problem of identifying the sparse principal component of a rank-deficient matrix. We introduce auxiliary spherical variables and prove that there exists a set of candidate index-sets (that is, sets of indices to the nonzero elements of the vector argument) whose size is polynomially bounded, in terms of rank, and contains the optimal index-set, i.e. the index-set of the nonzero ele…
▽ More
We consider the problem of identifying the sparse principal component of a rank-deficient matrix. We introduce auxiliary spherical variables and prove that there exists a set of candidate index-sets (that is, sets of indices to the nonzero elements of the vector argument) whose size is polynomially bounded, in terms of rank, and contains the optimal index-set, i.e. the index-set of the nonzero elements of the optimal solution. Finally, we develop an algorithm that computes the optimal sparse principal component in polynomial time for any sparsity degree.
△ Less
Submitted 8 June, 2011;
originally announced June 2011.
