-
A New Method for Employing Feedback to Improve Coding Performance
Authors:
Aaron B. Wagner,
Nirmal V. Shende,
Yücel Altuğ
Abstract:
We introduce a novel mechanism, called timid/bold coding, by which feedback can be used to improve coding performance. For a certain class of DMCs, called compound-dispersion channels, we show that timid/bold coding allows for an improved second-order coding rate compared with coding without feedback. For DMCs that are not compound dispersion, we show that feedback does not improve the second-orde…
▽ More
We introduce a novel mechanism, called timid/bold coding, by which feedback can be used to improve coding performance. For a certain class of DMCs, called compound-dispersion channels, we show that timid/bold coding allows for an improved second-order coding rate compared with coding without feedback. For DMCs that are not compound dispersion, we show that feedback does not improve the second-order coding rate. Thus we completely determine the class of DMCs for which feedback improves the second-order coding rate. An upper bound on the second-order coding rate is provided for compound-dispersion DMCs. We also show that feedback does not improve the second-order coding rate for very noisy DMCs. The main results are obtained by relating feedback codes to certain controlled diffusions.
△ Less
Submitted 29 August, 2019;
originally announced August 2019.
-
On exact asymptotics of the error probability in channel coding: symmetric channels
Authors:
Yücel Altuğ,
Aaron B. Wagner
Abstract:
The exact order of the optimal sub-exponentially decaying factor in the classical bounds on the error probability of fixed-length codes over a Gallager-symmetric discrete memoryless channel with and without ideal feedback is determined. Regardless of the availability of feedback, it is shown that the order of the optimal sub-exponential factor exhibits a dichotomy. Moreover, the proof technique is…
▽ More
The exact order of the optimal sub-exponentially decaying factor in the classical bounds on the error probability of fixed-length codes over a Gallager-symmetric discrete memoryless channel with and without ideal feedback is determined. Regardless of the availability of feedback, it is shown that the order of the optimal sub-exponential factor exhibits a dichotomy. Moreover, the proof technique is used to establish the third-order term in the normal approximation for symmetric channels, where a similar dichotomy is shown to exist.
△ Less
Submitted 29 August, 2019;
originally announced August 2019.
-
Information Theoretic Analysis of the Fundamental Limits of Content Identification
Authors:
Sait Tunc,
Yucel Altug,
Serdar Kozat,
Kivanc Mihcak
Abstract:
We investigate the content identification problem from an information theoretic perspective and derive its fundamental limits. Here, a rights-holder company desires to keep track of illegal uses of its commercial content, by utilizing resources of a security company, while securing the privacy of its content. Due to privacy issues, the rights-holder company only reveals certain hash values of the…
▽ More
We investigate the content identification problem from an information theoretic perspective and derive its fundamental limits. Here, a rights-holder company desires to keep track of illegal uses of its commercial content, by utilizing resources of a security company, while securing the privacy of its content. Due to privacy issues, the rights-holder company only reveals certain hash values of the original content to the security company. We view the commercial content of the rights-holder company as the codebook of an encoder and the hash values of the content (made available to the security company) as the codebook of a decoder, i.e., the corresponding codebooks of the encoder and the decoder are not the same. Hence, the content identification is modelled as a communication problem using asymmetric codebooks by an encoder and a decoder. We further address "the privacy issue" in the content identification by adding "security" constraints to the communication setup to prevent estimation of the encoder codewords given the decoder codewords. By this modeling, the proposed problem of reliable communication with asymmetric codebooks with security constraints provides the fundamental limits of the content identification problem. To this end, we introduce an information capacity and prove that this capacity is equal to the operation capacity of the system under i.i.d. encoder codewords providing the fundamental limits for content identification. As a well known and widely studied framework, we evaluate the capacity for a binary symmetric channel and provide closed form expressions.
△ Less
Submitted 7 October, 2018;
originally announced October 2018.
-
Minimax Rényi Redundancy
Authors:
Semih Yagli,
Yücel Altuğ,
Sergio Verdú
Abstract:
The redundancy for universal lossless compression of discrete memoryless sources in Campbell's setting is characterized as a minimax Rényi divergence, which is shown to be equal to the maximal $α$-mutual information via a generalized redundancy-capacity theorem. Special attention is placed on the analysis of the asymptotics of minimax Rényi divergence, which is determined up to a term vanishing in…
▽ More
The redundancy for universal lossless compression of discrete memoryless sources in Campbell's setting is characterized as a minimax Rényi divergence, which is shown to be equal to the maximal $α$-mutual information via a generalized redundancy-capacity theorem. Special attention is placed on the analysis of the asymptotics of minimax Rényi divergence, which is determined up to a term vanishing in blocklength.
△ Less
Submitted 24 January, 2018; v1 submitted 4 January, 2017;
originally announced January 2017.
-
Refinement of the random coding bound
Authors:
Yucel Altug,
Aaron B. Wagner
Abstract:
An improved pre-factor for the random coding bound is proved. Specifically, for channels with critical rate not equal to capacity, if a regularity condition is satisfied (resp. not satisfied), then for any $ε>0$ a pre-factor of $O(N^{-\frac{1}{2}\left( 1 - ε+ \barρ^\ast_R \right)})$ (resp. $O(N^{-\frac{1}{2}})$) is achievable for rates above the critical rate, where $N$ and $R$ is the blocklength…
▽ More
An improved pre-factor for the random coding bound is proved. Specifically, for channels with critical rate not equal to capacity, if a regularity condition is satisfied (resp. not satisfied), then for any $ε>0$ a pre-factor of $O(N^{-\frac{1}{2}\left( 1 - ε+ \barρ^\ast_R \right)})$ (resp. $O(N^{-\frac{1}{2}})$) is achievable for rates above the critical rate, where $N$ and $R$ is the blocklength and rate, respectively. The extra term $\barρ^\ast_R$ is related to the slope of the random coding exponent. Further, the relation of these bounds with the authors' recent refinement of the sphere-packing bound, as well as the pre-factor for the random coding bound below the critical rate, is discussed.
△ Less
Submitted 24 December, 2013;
originally announced December 2013.
-
The third-order term in the normal approximation for singular channels
Authors:
Yucel Altug,
Aaron B. Wagner
Abstract:
For a singular and symmetric discrete memoryless channel with positive dispersion, the third-order term in the normal approximation is shown to be upper bounded by a constant. This finding completes the characterization of the third-order term for symmetric discrete memoryless channels. The proof method is extended to asymmetric and singular channels with constant composition codes, and its connec…
▽ More
For a singular and symmetric discrete memoryless channel with positive dispersion, the third-order term in the normal approximation is shown to be upper bounded by a constant. This finding completes the characterization of the third-order term for symmetric discrete memoryless channels. The proof method is extended to asymmetric and singular channels with constant composition codes, and its connection to existing results, as well as its limitation in the error exponents regime, are discussed.
△ Less
Submitted 19 September, 2013;
originally announced September 2013.
-
Refinement of the Sphere-Packing Bound: Asymmetric Channels
Authors:
Yucel Altug,
Aaron B. Wagner
Abstract:
We provide a refinement of the sphere-packing bound for constant composition codes over asymmetric discrete memoryless channels that improves the pre-factor in front of the exponential term. The order of our pre-factor is $Ω(N^{-1/2(1+ε+ ρ_{R}^{\ast})})$ for any $ε>0$, where $ρ_{R}^{\ast}$ is the maximum absolute-value subdifferential of the sphere-packing exponent at rate $R$ and $N$ is the block…
▽ More
We provide a refinement of the sphere-packing bound for constant composition codes over asymmetric discrete memoryless channels that improves the pre-factor in front of the exponential term. The order of our pre-factor is $Ω(N^{-1/2(1+ε+ ρ_{R}^{\ast})})$ for any $ε>0$, where $ρ_{R}^{\ast}$ is the maximum absolute-value subdifferential of the sphere-packing exponent at rate $R$ and $N$ is the blocklength.
△ Less
Submitted 28 November, 2012;
originally announced November 2012.
-
Moderate Deviations in Channel Coding
Authors:
Yucel Altug,
Aaron B. Wagner
Abstract:
We consider block codes whose rate converges to the channel capacity with increasing block length at a certain speed and examine the best possible decay of the probability of error. We prove that a moderate deviation principle holds for all convergence rates between the large deviation and the central limit theorem regimes.
We consider block codes whose rate converges to the channel capacity with increasing block length at a certain speed and examine the best possible decay of the probability of error. We prove that a moderate deviation principle holds for all convergence rates between the large deviation and the central limit theorem regimes.
△ Less
Submitted 9 August, 2012;
originally announced August 2012.
-
On Detection With Partial Information In The Gaussian Setup
Authors:
Onur Ozyesil,
M. Kivanc Mihcak,
Yucel Altug
Abstract:
We introduce the problem of communication with partial information, where there is an asymmetry between the transmitter and the receiver codebooks. Practical applications of the proposed setup include the robust signal hashing problem within the context of multimedia security and asymmetric communications with resource-lacking receivers. We study this setup in a binary detection theoretic contex…
▽ More
We introduce the problem of communication with partial information, where there is an asymmetry between the transmitter and the receiver codebooks. Practical applications of the proposed setup include the robust signal hashing problem within the context of multimedia security and asymmetric communications with resource-lacking receivers. We study this setup in a binary detection theoretic context for the additive colored Gaussian noise channel. In our proposed setup, the partial information available at the detector consists of dimensionality-reduced versions of the transmitter codewords, where the dimensionality reduction is achieved via a linear transform. We first derive the corresponding MAP-optimal detection rule and the corresponding conditional probability of error (conditioned on the partial information the detector possesses). Then, we constructively quantify an optimal class of linear transforms, where the cost function is the expected Chernoff bound on the conditional probability of error of the MAP-optimal detector.
△ Less
Submitted 27 October, 2009;
originally announced October 2009.
-
Source and Channel Simulation Using Arbitrary Randomness
Authors:
Yucel Altug,
Aaron B. Wagner
Abstract:
Necessary and sufficient conditions for approximation of a general channel by a general source are proved. For the special case in which the channel input is deterministic, which corresponds to source simulation, we prove a stronger necessary condition. As the approximation criteria, vanishing variational distance between the original and the approximated quantity is used for both of the problem…
▽ More
Necessary and sufficient conditions for approximation of a general channel by a general source are proved. For the special case in which the channel input is deterministic, which corresponds to source simulation, we prove a stronger necessary condition. As the approximation criteria, vanishing variational distance between the original and the approximated quantity is used for both of the problems. Both necessary and sufficient conditions for the two problems are based on some individual properties of the sources and the channel and are relatively easy to evaluate. In particular, unlike prior results for this problem, our results do not require solving an optimization problem to test simulatability. The results are illustrated with several non-ergodic examples.
△ Less
Submitted 14 August, 2009;
originally announced August 2009.
-
An Information Theoretic Analysis of Single Transceiver Passive RFID Networks
Authors:
Yucel Altug,
S. Serdar Kozat,
M. Kivanc Mihcak
Abstract:
In this paper, we study single transceiver passive RFID networks by modeling the underlying physical system as a special cascade of a certain broadcast channel (BCC) and a multiple access channel (MAC), using a "nested codebook" structure in between. The particular application differentiates this communication setup from an ordinary cascade of a BCC and a MAC, and requires certain structures suc…
▽ More
In this paper, we study single transceiver passive RFID networks by modeling the underlying physical system as a special cascade of a certain broadcast channel (BCC) and a multiple access channel (MAC), using a "nested codebook" structure in between. The particular application differentiates this communication setup from an ordinary cascade of a BCC and a MAC, and requires certain structures such as "nested codebooks", impurity channels or additional power constraints. We investigate this problem both for discrete alphabets, where we characterize the achievable rate region, as well as for continuous alphabets with additive Gaussian noise, where we provide the capacity region. Hence, we establish the maximal achievable error free communication rates for this particular problem which constitutes the fundamental limit that is achievable by any TDMA based RFID protocol and the achievable rate region for any RFID protocol for the case of continuous alphabets under additive Gaussian noise.
△ Less
Submitted 6 January, 2009;
originally announced January 2009.
-
Reliable Communications with Asymmetric Codebooks: An Information Theoretic Analysis of Robust Signal Hashing
Authors:
Yucel Altug,
M. Kivanc Mihcak,
Onur Ozyesil,
Vishal Monga
Abstract:
In this paper, a generalization of the traditional point-to-point to communication setup, which is named as "reliable communications with asymmetric codebooks", is proposed. Under the assumption of independent identically distributed (i.i.d) encoder codewords, it is proven that the operational capacity of the system is equal to the information capacity of the system, which is given by…
▽ More
In this paper, a generalization of the traditional point-to-point to communication setup, which is named as "reliable communications with asymmetric codebooks", is proposed. Under the assumption of independent identically distributed (i.i.d) encoder codewords, it is proven that the operational capacity of the system is equal to the information capacity of the system, which is given by $\max_{p(x)} I(U;Y)$, where $X, U$ and $Y$ denote the individual random elements of encoder codewords, decoder codewords and decoder inputs. The capacity result is derived in the "binary symmetric" case (which is an analogous formulation of the traditional "binary symmetric channel" for our case), as a function of the system parameters. A conceptually insightful inference is made by attributing the difference from the classical Shannon-type capacity of binary symmetric channel to the {\em gap} due to the codebook asymmetry.
△ Less
Submitted 10 September, 2008;
originally announced September 2008.
-
Towards Exploring Fundamental Limits of System-Specific Cryptanalysis Within Limited Attack Classes: Application to ABSG
Authors:
Yucel Altug,
M. Kivanc Mihcak
Abstract:
A new approach on cryptanalysis is proposed where the goal is to explore the fundamental limits of a specific class of attacks against a particular cryptosystem. As a first step, the approach is applied on ABSG, which is an LFSR-based stream cipher where irregular decimation techniques are utilized. Consequently, under some mild assumptions, which are common in cryptanalysis, the tight lower bou…
▽ More
A new approach on cryptanalysis is proposed where the goal is to explore the fundamental limits of a specific class of attacks against a particular cryptosystem. As a first step, the approach is applied on ABSG, which is an LFSR-based stream cipher where irregular decimation techniques are utilized. Consequently, under some mild assumptions, which are common in cryptanalysis, the tight lower bounds on the algorithmic complexity of successful Query-Based Key-Recovery attacks are derived for two different setups of practical interest. The proofs rely on the concept of ``typicality'' of information theory.
△ Less
Submitted 2 May, 2008;
originally announced May 2008.
-
A Note on the Periodicity and the Output Rate of Bit Search Type Generators
Authors:
Yucel Altug,
N. Polat Ayerden,
M. Kivanc Mihcak,
Emin Anarim
Abstract:
We investigate the bit-search type irregular decimation algorithms that are used within LFSR-based stream ciphers. In particular, we concentrate on BSG and ABSG, and consider two different setups for the analysis. In the first case, the input is assumed to be a m-sequence; we show that all possible output sequences can be classified into two sets, each of which is characterized by the equivalenc…
▽ More
We investigate the bit-search type irregular decimation algorithms that are used within LFSR-based stream ciphers. In particular, we concentrate on BSG and ABSG, and consider two different setups for the analysis. In the first case, the input is assumed to be a m-sequence; we show that all possible output sequences can be classified into two sets, each of which is characterized by the equivalence of their elements up to shifts. Furthermore, we prove that the cardinality of each of these sets is equal to the period of one of its elements and subsequently derive the first known bounds on the expected output period (assuming that no subperiods exist). In the second setup, we work in a probabilistic framework and assume that the input sequence is evenly distributed (i.e., independent identically distributed Bernoulli process with probability 1/2). Under these assumptions, we derive closed-form expressions for the distribution of the output length and the output rate, which is shown to be asymptotically Gaussian-distributed and concentrated around the mean with exponential tightness.
△ Less
Submitted 16 February, 2007;
originally announced February 2007.
