-
Computational Concentration of Measure: Optimal Bounds, Reductions, and More
Authors:
Omid Etesami,
Saeed Mahloujifar,
Mohammad Mahmoody
Abstract:
Product measures of dimension $n$ are known to be concentrated in Hamming distance: for any set $S$ in the product space of probability $ε$, a random point in the space, with probability $1-δ$, has a neighbor in $S$ that is different from the original point in only $O(\sqrt{n\ln(1/(εδ))})$ coordinates. We obtain the tight computational version of this result, showing how given a random point and a…
▽ More
Product measures of dimension $n$ are known to be concentrated in Hamming distance: for any set $S$ in the product space of probability $ε$, a random point in the space, with probability $1-δ$, has a neighbor in $S$ that is different from the original point in only $O(\sqrt{n\ln(1/(εδ))})$ coordinates. We obtain the tight computational version of this result, showing how given a random point and access to an $S$-membership oracle, we can find such a close point in polynomial time. This resolves an open question of [Mahloujifar and Mahmoody, ALT 2019]. As corollaries, we obtain polynomial-time poisoning and (in certain settings) evasion attacks against learning algorithms when the original vulnerabilities have any cryptographically non-negligible probability.
We call our algorithm MUCIO ("MUltiplicative Conditional Influence Optimizer") since proceeding through the coordinates, it decides to change each coordinate of the given point based on a multiplicative version of the influence of that coordinate, where influence is computed conditioned on previously updated coordinates.
We also define a new notion of algorithmic reduction between computational concentration of measure in different metric probability spaces. As an application, we get computational concentration of measure for high-dimensional Gaussian distributions under the $\ell_1$ metric.
We prove several extensions to the results above: (1) Our computational concentration result is also true when the Hamming distance is weighted. (2) We obtain an algorithmic version of concentration around mean, more specifically, McDiarmid's inequality. (3) Our result generalizes to discrete random processes, and this leads to new tampering algorithms for collective coin tossing protocols. (4) We prove exponential lower bounds on the average running time of non-adaptive query algorithms.
△ Less
Submitted 11 July, 2019;
originally announced July 2019.
-
Complete Classification of Generalized Santha-Vazirani Sources
Authors:
Salman Beigi,
Andrej Bogdanov,
Omid Etesami,
Siyao Guo
Abstract:
Let $\mathcal{F}$ be a finite alphabet and $\mathcal{D}$ be a finite set of distributions over $\mathcal{F}$. A Generalized Santha-Vazirani (GSV) source of type $(\mathcal{F}, \mathcal{D})$, introduced by Beigi, Etesami and Gohari (ICALP 2015, SICOMP 2017), is a random sequence $(F_1, \dots, F_n)$ in $\mathcal{F}^n$, where $F_i$ is a sample from some distribution $d \in \mathcal{D}$ whose choice m…
▽ More
Let $\mathcal{F}$ be a finite alphabet and $\mathcal{D}$ be a finite set of distributions over $\mathcal{F}$. A Generalized Santha-Vazirani (GSV) source of type $(\mathcal{F}, \mathcal{D})$, introduced by Beigi, Etesami and Gohari (ICALP 2015, SICOMP 2017), is a random sequence $(F_1, \dots, F_n)$ in $\mathcal{F}^n$, where $F_i$ is a sample from some distribution $d \in \mathcal{D}$ whose choice may depend on $F_1, \dots, F_{i-1}$.
We show that all GSV source types $(\mathcal{F}, \mathcal{D})$ fall into one of three categories: (1) non-extractable; (2) extractable with error $n^{-Θ(1)}$; (3) extractable with error $2^{-Ω(n)}$. This rules out other error rates like $1/\log n$ or $2^{-\sqrt{n}}$.
We provide essentially randomness-optimal extraction algorithms for extractable sources. Our algorithm for category (2) sources extracts with error $\varepsilon$ from $n = \mathrm{poly}(1/\varepsilon)$ samples in time linear in $n$. Our algorithm for category (3) sources extracts $m$ bits with error $\varepsilon$ from $n = O(m + \log 1/\varepsilon)$ samples in time $\min\{O(nm2^m),n^{O(\lvert\mathcal{F}\rvert)}\}$.
We also give algorithms for classifying a GSV source type $(\mathcal{F}, \mathcal{D})$: Membership in category (1) can be decided in $\mathrm{NP}$, while membership in category (3) is polynomial-time decidable.
△ Less
Submitted 10 September, 2017;
originally announced September 2017.
-
On the Monotone Measure of Correlation
Authors:
Omid Etesami,
Amin Gohari
Abstract:
Based on the notion of maximal correlation, Kimeldorf, May and Sampson (1980) introduce a measure of correlation between two random variables, called the "concordant monotone correlation" (CMC). We revisit, generalize and prove new properties of this measure of correlation. It is shown that CMC captures various types of correlation detected in measures of rank correlation like the Kendall tau corr…
▽ More
Based on the notion of maximal correlation, Kimeldorf, May and Sampson (1980) introduce a measure of correlation between two random variables, called the "concordant monotone correlation" (CMC). We revisit, generalize and prove new properties of this measure of correlation. It is shown that CMC captures various types of correlation detected in measures of rank correlation like the Kendall tau correlation. We show that the CMC satisfies the data processing and tensorization properties (that make ordinary maximal correlation applicable to problems in information theory). Furthermore, CMC is shown to be intimately related to the FKG inequality. Furthermore, a combinatorical application of CMC is given for which we do not know of another method to derive its result. Finally, we study the problem of the complexity of the computation of the CMC, which is a non-convex optimization problem with local maximas. We give a simple but exponential-time algorithm that is guaranteed to output the exact value of the generalized CMC.
△ Less
Submitted 22 June, 2016; v1 submitted 13 November, 2015;
originally announced November 2015.
-
Deterministic Randomness Extraction from Generalized and Distributed Santha-Vazirani Sources
Authors:
Salman Beigi,
Omid Etesami,
Amin Gohari
Abstract:
A Santha-Vazirani (SV) source is a sequence of random bits where the conditional distribution of each bit, given the previous bits, can be partially controlled by an adversary. Santha and Vazirani show that deterministic randomness extraction from these sources is impossible. In this paper, we study the generalization of SV sources for non-binary sequences. We show that unlike the binary case, det…
▽ More
A Santha-Vazirani (SV) source is a sequence of random bits where the conditional distribution of each bit, given the previous bits, can be partially controlled by an adversary. Santha and Vazirani show that deterministic randomness extraction from these sources is impossible. In this paper, we study the generalization of SV sources for non-binary sequences. We show that unlike the binary case, deterministic randomness extraction in the generalized case is sometimes possible. We present a necessary condition and a sufficient condition for the possibility of deterministic randomness extraction. These two conditions coincide in "non-degenerate" cases.
Next, we turn to a distributed setting. In this setting the SV source consists of a random sequence of pairs $(a_1, b_1), (a_2, b_2), \ldots$ distributed between two parties, where the first party receives $a_i$'s and the second one receives $b_i$'s. The goal of the two parties is to extract common randomness without communication. Using the notion of maximal correlation, we prove a necessary condition and a sufficient condition for the possibility of common randomness extraction from these sources. Based on these two conditions, the problem of common randomness extraction essentially reduces to the problem of randomness extraction from (non-distributed) SV sources. This result generalizes results of Gács and Körner, and Witsenhausen about common randomness extraction from i.i.d. sources to adversarial sources.
△ Less
Submitted 20 December, 2014;
originally announced December 2014.
-
The Value of Help Bits in Randomized and Average-Case Complexity
Authors:
Salman Beigi,
Omid Etesami,
Amin Gohari
Abstract:
"Help bits" are some limited trusted information about an instance or instances of a computational problem that may reduce the computational complexity of solving that instance or instances. In this paper, we study the value of help bits in the settings of randomized and average-case complexity.
Amir, Beigel, and Gasarch (1990) show that for constant $k$, if $k$ instances of a decision problem c…
▽ More
"Help bits" are some limited trusted information about an instance or instances of a computational problem that may reduce the computational complexity of solving that instance or instances. In this paper, we study the value of help bits in the settings of randomized and average-case complexity.
Amir, Beigel, and Gasarch (1990) show that for constant $k$, if $k$ instances of a decision problem can be efficiently solved using less than $k$ bits of help, then the problem is in P/poly. We extend this result to the setting of randomized computation: We show that the decision problem is in P/poly if using $\ell$ help bits, $k$ instances of the problem can be efficiently solved with probability greater than $2^{\ell-k}$. The same result holds if using less than $k(1 - h(α))$ help bits (where $h(\cdot)$ is the binary entropy function), we can efficiently solve $(1-α)$ fraction of the instances correctly with non-vanishing probability. We also extend these two results to non-constant but logarithmic $k$. In this case however, instead of showing that the problem is in P/poly we show that it satisfies "$k$-membership comparability," a notion known to be related to solving $k$ instances using less than $k$ bits of help.
Next we consider the setting of average-case complexity: Assume that we can solve $k$ instances of a decision problem using some help bits whose entropy is less than $k$ when the $k$ instances are drawn independently from a particular distribution. Then we can efficiently solve an instance drawn from that distribution with probability better than $1/2$.
Finally, we show that in the case where $k$ is super-logarithmic, assuming $k$-membership comparability of a decision problem, one cannot prove that the problem is in P/poly by a "black-box proof."
△ Less
Submitted 3 August, 2014;
originally announced August 2014.
-
Irregular Product Codes
Authors:
Masoud Alipour,
Omid Etesami,
Ghid Maatouk,
Amin Shokrollahi
Abstract:
We consider irregular product codes.In this class of codes, each codeword is represented by a matrix. The entries in each row (column) of the matrix should come from a component row (column) code. As opposed to (standard) product codes, we do not require that all component row codes nor all component column codes be the same. As we will see, relaxing this requirement can provide some additional at…
▽ More
We consider irregular product codes.In this class of codes, each codeword is represented by a matrix. The entries in each row (column) of the matrix should come from a component row (column) code. As opposed to (standard) product codes, we do not require that all component row codes nor all component column codes be the same. As we will see, relaxing this requirement can provide some additional attractive features including 1) allowing some regions of the codeword be more error-resilient 2) allowing a more refined spectrum of rates for finite-lengths and improved performance in some of these rates 3) more interaction between row and column codes during decoding. We study these codes over erasure channels. We find that for any $0 < ε< 1$, for many rate distributions on component row codes, there is a matching rate distribution on component column codes such that an irregular product code based on MDS codes with those rate distributions on the component codes has asymptotic rate $1 - ε$ and can decode on erasure channels (of alphabet size equal the alphabet size of the component MDS codes) with erasure probability $< ε$.
△ Less
Submitted 11 June, 2012;
originally announced June 2012.
