-
Unbounded Error Correcting Codes
Authors:
Klim Efremenko,
Or Zamir
Abstract:
Traditional error-correcting codes (ECCs) assume a fixed message length, but many scenarios involve ongoing or indefinite transmissions where the message length is not known in advance. For example, when streaming a video, the user should be able to fix a fraction of errors that occurred before any point in time. We introduce unbounded error-correcting codes (unbounded codes), a natural generaliza…
▽ More
Traditional error-correcting codes (ECCs) assume a fixed message length, but many scenarios involve ongoing or indefinite transmissions where the message length is not known in advance. For example, when streaming a video, the user should be able to fix a fraction of errors that occurred before any point in time. We introduce unbounded error-correcting codes (unbounded codes), a natural generalization of ECCs that supports arbitrarily long messages without a predetermined length. An unbounded code with rate $R$ and distance $\varepsilon$ ensures that for every sufficiently large $k$, the message prefix of length $Rk$ can be recovered from the code prefix of length $k$ even if an adversary corrupts up to an $\varepsilon$ fraction of the symbols in this code prefix.
We study unbounded codes over binary alphabets in the regime of small error fraction $\varepsilon$, establishing nearly tight upper and lower bounds on their optimal rate. Our main results show that: (1) The optimal rate of unbounded codes satisfies $R<1-Ω(\sqrt{\varepsilon})$ and $R>1-O(\sqrt{\varepsilon \log \log(1/\varepsilon)})$. (2) Surprisingly, our construction is inherently non-linear, as we prove that linear unbounded codes achieve a strictly worse rate of $R=1-Θ(\sqrt{\varepsilon \log(1/\varepsilon)})$. (3) In the setting of random noise, unbounded codes achieve the same optimal rate as standard ECCs, $R=1-Θ(\varepsilon \log(1/\varepsilon))$.
These results demonstrate fundamental differences between standard and unbounded codes.
△ Less
Submitted 8 April, 2025; v1 submitted 7 November, 2024;
originally announced November 2024.
-
Statistically Near-Optimal Hypothesis Selection
Authors:
Olivier Bousquet,
Mark Braverman,
Klim Efremenko,
Gillat Kol,
Shay Moran
Abstract:
Hypothesis Selection is a fundamental distribution learning problem where given a comparator-class $Q=\{q_1,\ldots, q_n\}$ of distributions, and a sampling access to an unknown target distribution $p$, the goal is to output a distribution $q$ such that $\mathsf{TV}(p,q)$ is close to $opt$, where $opt = \min_i\{\mathsf{TV}(p,q_i)\}$ and $\mathsf{TV}(\cdot, \cdot)$ denotes the total-variation distan…
▽ More
Hypothesis Selection is a fundamental distribution learning problem where given a comparator-class $Q=\{q_1,\ldots, q_n\}$ of distributions, and a sampling access to an unknown target distribution $p$, the goal is to output a distribution $q$ such that $\mathsf{TV}(p,q)$ is close to $opt$, where $opt = \min_i\{\mathsf{TV}(p,q_i)\}$ and $\mathsf{TV}(\cdot, \cdot)$ denotes the total-variation distance. Despite the fact that this problem has been studied since the 19th century, its complexity in terms of basic resources, such as number of samples and approximation guarantees, remains unsettled (this is discussed, e.g., in the charming book by Devroye and Lugosi `00). This is in stark contrast with other (younger) learning settings, such as PAC learning, for which these complexities are well understood.
We derive an optimal $2$-approximation learning strategy for the Hypothesis Selection problem, outputting $q$ such that $\mathsf{TV}(p,q) \leq2 \cdot opt + \eps$, with a (nearly) optimal sample complexity of~$\tilde O(\log n/ε^2)$. This is the first algorithm that simultaneously achieves the best approximation factor and sample complexity: previously, Bousquet, Kane, and Moran (COLT `19) gave a learner achieving the optimal $2$-approximation, but with an exponentially worse sample complexity of $\tilde O(\sqrt{n}/ε^{2.5})$, and Yatracos~(Annals of Statistics `85) gave a learner with optimal sample complexity of $O(\log n /ε^2)$ but with a sub-optimal approximation factor of $3$.
△ Less
Submitted 17 August, 2021;
originally announced August 2021.
-
Fast and Bayes-consistent nearest neighbors
Authors:
Klim Efremenko,
Aryeh Kontorovich,
Moshe Noivirt
Abstract:
Research on nearest-neighbor methods tends to focus somewhat dichotomously either on the statistical or the computational aspects -- either on, say, Bayes consistency and rates of convergence or on techniques for speeding up the proximity search. This paper aims at bridging these realms: to reap the advantages of fast evaluation time while maintaining Bayes consistency, and further without sacrifi…
▽ More
Research on nearest-neighbor methods tends to focus somewhat dichotomously either on the statistical or the computational aspects -- either on, say, Bayes consistency and rates of convergence or on techniques for speeding up the proximity search. This paper aims at bridging these realms: to reap the advantages of fast evaluation time while maintaining Bayes consistency, and further without sacrificing too much in the risk decay rate. We combine the locality-sensitive hashing (LSH) technique with a novel missing-mass argument to obtain a fast and Bayes-consistent classifier. Our algorithm's prediction runtime compares favorably against state of the art approximate NN methods, while maintaining Bayes-consistency and attaining rates comparable to minimax. On samples of size $n$ in $\R^d$, our pre-processing phase has runtime $O(d n \log n)$, while the evaluation phase has runtime $O(d\log n)$ per query point.
△ Less
Submitted 15 April, 2020; v1 submitted 7 October, 2019;
originally announced October 2019.
-
The method of shifted partial derivatives cannot separate the permanent from the determinant
Authors:
Klim Efremenko,
J. M. Landsberg,
Hal Schenck,
Jerzy Weyman
Abstract:
The method of shifted partial derivatives was used to prove a super-polynomial lower bound on the size of depth four circuits needed to compute the permanent. We show that this method alone cannot prove that the padded permanent $\ell^{n-m} perm_m$ cannot be realized inside the $GL_{n^2}$-orbit closure of the determinant $ det_n$ when $n>2m^2+2m$. Our proof relies on several simple degenerations o…
▽ More
The method of shifted partial derivatives was used to prove a super-polynomial lower bound on the size of depth four circuits needed to compute the permanent. We show that this method alone cannot prove that the padded permanent $\ell^{n-m} perm_m$ cannot be realized inside the $GL_{n^2}$-orbit closure of the determinant $ det_n$ when $n>2m^2+2m$. Our proof relies on several simple degenerations of the determinant polynomial, Macaulay's theorem that gives a lower bound on the growth of an ideal, and a lower bound estimate from Gupta et. al. regarding the shifted partial derivatives of the determinant.
△ Less
Submitted 7 September, 2016;
originally announced September 2016.
-
Testing Equality in Communication Graphs
Authors:
Noga Alon,
Klim Efremenko,
Benny Sudakov
Abstract:
Let $G=(V,E)$ be a connected undirected graph with $k$ vertices. Suppose that on each vertex of the graph there is a player having an $n$-bit string. Each player is allowed to communicate with its neighbors according to an agreed communication protocol, and the players must decide, deterministically, if their inputs are all equal. What is the minimum possible total number of bits transmitted in a…
▽ More
Let $G=(V,E)$ be a connected undirected graph with $k$ vertices. Suppose that on each vertex of the graph there is a player having an $n$-bit string. Each player is allowed to communicate with its neighbors according to an agreed communication protocol, and the players must decide, deterministically, if their inputs are all equal. What is the minimum possible total number of bits transmitted in a protocol solving this problem ? We determine this minimum up to a lower order additive term in many cases (but not for all graphs). In particular, we show that it is $kn/2+o(n)$ for any Hamiltonian $k$-vertex graph, and that for any $2$-edge connected graph with $m$ edges containing no two adjacent vertices of degree exceeding $2$ it is $mn/2+o(n)$. The proofs combine graph theoretic ideas with tools from additive number theory.
△ Less
Submitted 5 May, 2016;
originally announced May 2016.
-
On minimal free resolutions of sub-permanents and other ideals arising in complexity theory
Authors:
Klim Efremenko,
J. M. Landsberg,
Hal Schenck,
Jerzy Weyman
Abstract:
We compute the linear strand of the minimal free resolution of the ideal generated by k x k sub-permanents of an n x n generic matrix and of the ideal generated by square-free monomials of degree k. The latter calculation gives the full minimal free resolution by work of Biagioli-Faridi-Rosas. Our motivation is to lay groundwork for the use of commutative algebra in algebraic complexity theory. We…
▽ More
We compute the linear strand of the minimal free resolution of the ideal generated by k x k sub-permanents of an n x n generic matrix and of the ideal generated by square-free monomials of degree k. The latter calculation gives the full minimal free resolution by work of Biagioli-Faridi-Rosas. Our motivation is to lay groundwork for the use of commutative algebra in algebraic complexity theory. We also compute several Hilbert functions relevant for complexity theory.
△ Less
Submitted 3 December, 2017; v1 submitted 20 April, 2015;
originally announced April 2015.
