-
On the Fragile Rates of Linear Feedback Coding Schemes of Gaussian Channels with Memory
Authors:
Charalambos D. Charalambous,
Christos Kourtellaris,
Themistoklis Charalambous
Abstract:
In \cite{butman1976} the linear coding scheme is applied, $X_t =g_t\Big(Θ- {\bf E}\Big\{Θ\Big|Y^{t-1}, V_0=v_0\Big\}\Big)$, $t=2,\ldots,n$, $X_1=g_1Θ$, with $Θ: Ω\to {\mathbb R}$, a Gaussian random variable, to derive a lower bound on the feedback rate, for additive Gaussian noise (AGN) channels, $Y_t=X_t+V_t, t=1, \ldots, n$, where $V_t$ is a Gaussian autoregressive (AR) noise, and…
▽ More
In \cite{butman1976} the linear coding scheme is applied, $X_t =g_t\Big(Θ- {\bf E}\Big\{Θ\Big|Y^{t-1}, V_0=v_0\Big\}\Big)$, $t=2,\ldots,n$, $X_1=g_1Θ$, with $Θ: Ω\to {\mathbb R}$, a Gaussian random variable, to derive a lower bound on the feedback rate, for additive Gaussian noise (AGN) channels, $Y_t=X_t+V_t, t=1, \ldots, n$, where $V_t$ is a Gaussian autoregressive (AR) noise, and $κ\in [0,\infty)$ is the total transmitter power. For the unit memory AR noise, with parameters $(c, K_W)$, where $c\in [-1,1]$ is the pole and $K_W$ is the variance of the Gaussian noise, the lower bound is $C^{L,B} =\frac{1}{2} \log χ^2$, where $χ=\lim_{n\longrightarrow \infty} χ_n$ is the positive root of $χ^2=1+\Big(1+ \frac{|c|}χ\Big)^2 \fracκ{K_W}$, and the sequence $χ_n \triangleq \Big|\frac{g_n}{g_{n-1}}\Big|, n=2, 3, \ldots,$ satisfies a certain recursion, and conjectured that $C^{L,B}$ is the feedback capacity.
In this correspondence, it is observed that the nontrivial lower bound $C^{L,B}=\frac{1}{2} \log χ^2$ such that $χ>1$, necessarily implies the scaling coefficients of the feedback code, $g_n$, $n=1,2, \ldots$, grow unbounded, in the sense that, $\lim_{n\longrightarrow\infty}|g_n| =+\infty$. The unbounded behaviour of $g_n$ follows from the ratio limit theorem of a sequence of real numbers, and it is verified by simulations. It is then concluded that such linear codes are not practical, and fragile with respect to a mismatch between the statistics of the mathematical model of the channel and the real statistics of the channel. In particular, if the error is perturbed by $ε_n>0$ no matter how small, then $X_n =g_t\Big(Θ- {\bf E}\Big\{Θ\Big|Y^{t-1}, V_0=v_0\Big\}\Big)+g_n ε_n$, and $|g_n|ε_n \longrightarrow \infty$, as $n \longrightarrow \infty$.
△ Less
Submitted 16 June, 2021;
originally announced June 2021.
-
On the Cover and Pombra Gaussian Feedback Capacity: Complete Sequential Characterizations via a Sufficient Statistic
Authors:
Charalambos D. Charalambous,
Christos Kourtellaris,
Stelios Louka
Abstract:
The main objective of this paper is to derive a new sequential characterization of the Cover and Pombra \cite{cover-pombra1989} characterization of the $n-$finite block or transmission feedback information ($n$-FTFI) capacity, which clarifies several issues of confusion and incorrect interpretation of results in literature. The optimal channel input processes of the new equivalent sequential chara…
▽ More
The main objective of this paper is to derive a new sequential characterization of the Cover and Pombra \cite{cover-pombra1989} characterization of the $n-$finite block or transmission feedback information ($n$-FTFI) capacity, which clarifies several issues of confusion and incorrect interpretation of results in literature. The optimal channel input processes of the new equivalent sequential characterizations are expressed as functionals of a sufficient statistic and a Gaussian orthogonal innovations process. From the new representations follows that the Cover and Pombra characterization of the $n-$FTFI capacity is expressed as a functional of two generalized matrix difference Riccati equations (DRE) of filtering theory of Gaussian systems. This contradicts results which are redundant in the literature, and illustrates the fundamental complexity of the feedback capacity formula.
△ Less
Submitted 9 June, 2021;
originally announced June 2021.
-
Feedback Capacity Formulas of AGN Channels Driven by Nonstationary Autoregressive Moving Average Noise
Authors:
Stelios Louka,
Christos Kourtellaris,
Charalambos D. Charalambous
Abstract:
In this paper we derive closed-form formulas of feedback capacity and nonfeedback achievable rates, for Additive Gaussian Noise (AGN) channels driven by nonstationary autoregressive moving average (ARMA) noise (with unstable one poles and zeros), based on time-invariant feedback codes and channel input distributions. From the analysis and simulations follows the surprising observations, (i) the us…
▽ More
In this paper we derive closed-form formulas of feedback capacity and nonfeedback achievable rates, for Additive Gaussian Noise (AGN) channels driven by nonstationary autoregressive moving average (ARMA) noise (with unstable one poles and zeros), based on time-invariant feedback codes and channel input distributions. From the analysis and simulations follows the surprising observations, (i) the use of time-invariant channel input distributions gives rise to multiple regimes of capacity that depend on the parameters of the ARMA noise, which may or may not use feedback, (ii) the more unstable the pole (resp. zero) of the ARMA noise the higher (resp. lower) the feedback capacity, (iii) certain conditions, known as detectability and stabilizability are necessary and sufficient to ensure the feedback capacity formulas and nonfeedback achievable rates {\it are independent of the initial state of the ARMA noise}. Another surprizing observation is that Kim's \cite{kim2010} characterization of feedback capacity which is developed for stable ARMA noise, if applied to the unstable ARMA noise, gives a lower value of feedback capacity compared to our feedback capacity formula.
△ Less
Submitted 9 June, 2021;
originally announced June 2021.
-
New Formulas of Feedback Capacity for AGN Channels with Memory: A Time-Domain Sufficient Statistic Approach
Authors:
Charalambos D. Charalambous,
Christos Kourtellaris,
Stelios Louka
Abstract:
In the recent paper [1] it is shown, via an application example, that the Cover and Pombra [2] "characterization of the $n-$block or transmission" feedback capacity formula, of additive Gaussian noise (AGN) channels, is the subject of much confusion in the literature, with redundant incorrect results. The main objective of this paper is to clarify the main points of confusion and remove any furthe…
▽ More
In the recent paper [1] it is shown, via an application example, that the Cover and Pombra [2] "characterization of the $n-$block or transmission" feedback capacity formula, of additive Gaussian noise (AGN) channels, is the subject of much confusion in the literature, with redundant incorrect results. The main objective of this paper is to clarify the main points of confusion and remove any further ambiguity. The first part of the paper applies time-domain methods, to derive for a first time, equivalent sequential characterizations of the Cover and Pombra characterization of feedback capacity of AGN channels driven by nonstationary and nonergodic Gaussian noise. The optimal channel input processes of the new equivalent sequential characterizations are expressed as functionals of a sufficient statistic and a Gaussian orthogonal innovations process. From the new representations follows that the Cover and Pombra $n-$block capacity formula is expressed as a functional of two generalized matrix difference Riccati equations (DRE) of filtering theory of Gaussian systems, contrary to results that appeared in the literature. In the second part of the paper the existence of the asymptotic limit of the $n-$block feedback capacity formula is shown to be equivalent to the convergence properties of solutions of the two generalized DREs. Further, necessary and or sufficient conditions are identified for existence of the asymptotic limits, for stable and unstable Gaussian noise, when the optimal input distributions are time-invariant, but not necessarily stationary. The paper contains an in depth analysis, with examples, of the specific technical issues, which are overlooked in past literature [3-7], that studied the AGN channel of [2], for stationary noises.
△ Less
Submitted 13 October, 2020;
originally announced October 2020.
-
Time-Invariant Feedback Strategies Do Not Increase Capacity of AGN Channels Driven by Stable and Certain Unstable Autoregressive Noise
Authors:
Charalambos D. Charalambous,
Christos Kourtellaris,
Sergey Loyka
Abstract:
The capacity of additive Gaussian noise (AGN) channels, $Y_t=X_t+V_t, t=1, \ldots, n$, $\frac{1}{n} {\bf E}\big\{\sum_{t=1}^n |X_t|^2 \big\}\leq κ, κ\in [0,\infty)$, with time-invariant channel input feedback strategies, is characterized and conditions are identified for entropy rates, and limit of average power to exist, when the noise is described by {\it stable and unstable} autoregressive mode…
▽ More
The capacity of additive Gaussian noise (AGN) channels, $Y_t=X_t+V_t, t=1, \ldots, n$, $\frac{1}{n} {\bf E}\big\{\sum_{t=1}^n |X_t|^2 \big\}\leq κ, κ\in [0,\infty)$, with time-invariant channel input feedback strategies, is characterized and conditions are identified for entropy rates, and limit of average power to exist, when the noise is described by {\it stable and unstable} autoregressive models, AR$(c)$, $V_t=cV_{t-1}+ W_t, V_0=v_0, t=1, \ldots, n$, where $c\in (-\infty,\infty)$, $W_t, t=1,\ldots, n$, is a zero mean, variance $K_W$, independent Gaussian sequence, independent of $V_0$. For stable AR$(c), c\in (-1,1)$ the conditions are necessary and sufficient for asymptotic stationarity of the processes $(X_t, Y_t), t=1, 2, \ldots$. New closed form capacity formulas and lower bounds are derived, for the AR$(c), c\in (-\infty,\infty)$ noise, which are fundamentally different from existing formulas in the literature, and illustrate multiple regimes of capacity, as a function of the parameters $(c,K_W,κ)$, as follows.\\ 1) feedback increases capacity for the regime, $c^2 \in (1, \infty),$ for $κ> \frac{K_W\big(1+\sqrt{4c^2-3}\big)}{2\big(c^2-1\big)^2}$, \\ 2) feedback does not increase capacity for the regime $c^2 \in (1, \infty)$, for $κ\leq \frac{K_W\big(1+\sqrt{4c^2-3}\big)}{2\big(c^2-1\big)^2}$, and \\ 3) feedback does not increase capacity for the regime $c \in [-1,1]$, for $ κ\in [0,\infty)$.
△ Less
Submitted 13 August, 2021; v1 submitted 25 July, 2019;
originally announced July 2019.
-
Finite Blocklength Analysis of Multiple Access Channels with/without Cooperation
Authors:
Christos K. Kourtellaris,
Constantinos Psomas,
Ioannis Krikidis
Abstract:
Motivated by the demand of reliable and low latency communications, we employ tools from information theory, stochastic processes and queueing theory, in order to provide a comprehensive framework regarding the analysis of a Time Division Multiple Access (TDMA) network with bursty traffic, in the finite blocklength regime. Specifically, we re-examine the stability conditions of a noncooperative TD…
▽ More
Motivated by the demand of reliable and low latency communications, we employ tools from information theory, stochastic processes and queueing theory, in order to provide a comprehensive framework regarding the analysis of a Time Division Multiple Access (TDMA) network with bursty traffic, in the finite blocklength regime. Specifically, we re-examine the stability conditions of a noncooperative TDMA multiple access channel, evaluate the optimal throughput, and identify the optimal trade-off between data packet size and latency. The evaluation is performed both numerically and via the proposed approximations that result in closed form expressions. Then, we examine the stability conditions and the performance of the Multiple Access Relay Channel with TDMA scheduling, subject to finite blocklength constraints, by applying a cognitive cooperation protocol that assumes relaying is enabled when sources are idle. Finally, we propose the novel Batch-And-Forward (BAF) strategy, that can significantly enhance the performance of cooperative networks in the finite blocklength regime, as well as reduce the requirement in metadata. The BAF strategy is quite versatile, thus, it can be embedded in existing cooperative protocols, without imposing additional complexity on the overall scheme.
△ Less
Submitted 17 September, 2019; v1 submitted 6 August, 2018;
originally announced August 2018.
-
Single Letter Expression of Capacity for a Class of Channels with Memory
Authors:
Christos K. Kourtellaris,
Charalambos D. Charalambous,
Ioannis Tzortzis
Abstract:
We study finite alphabet channels with Unit Memory on the previous Channel Outputs called UMCO channels. We identify necessary and sufficient conditions, to test whether the capacity achieving channel input distributions with feedback are time-invariant, and whether feedback capacity is characterized by single letter, expressions, similar to that of memoryless channels. The method is based on show…
▽ More
We study finite alphabet channels with Unit Memory on the previous Channel Outputs called UMCO channels. We identify necessary and sufficient conditions, to test whether the capacity achieving channel input distributions with feedback are time-invariant, and whether feedback capacity is characterized by single letter, expressions, similar to that of memoryless channels. The method is based on showing that a certain dynamic programming equation, which in general, is a nested optimization problem over the sequence of channel input distributions, reduces to a non-nested optimization problem. Moreover, for UMCO channels, we give a simple expression for the ML error exponent, and we identify sufficient conditions to test whether feedback does not increase capacity. We derive similar results, when transmission cost constraints are imposed. We apply the results to a special class of the UMCO channels, the Binary State Symmetric Channel (BSSC) with and without transmission cost constraints, to show that the optimization problem of feedback capacity is non-nested, the capacity achieving channel input distribution and the corresponding channel output transition probability distribution are time-invariant, and feedback capacity is characterized by a single letter formulae, precisely as Shannon's single letter characterization of capacity of memoryless channels. Then we derive closed form expressions for the capacity achieving channel input distribution and feedback capacity. We use the closed form expressions to evaluate an error exponent for ML decoding.
△ Less
Submitted 4 January, 2017;
originally announced January 2017.
-
Sequential Necessary and Sufficient Conditions for Capacity Achieving Distributions of Channels with Memory and Feedback
Authors:
Photios A. Stavrou,
Charalambos D. Charalambous,
Christos K. Kourtellaris
Abstract:
We derive sequential necessary and sufficient conditions for any channel input conditional distribution ${\cal P}_{0,n}\triangleq\{P_{X_t|X^{t-1},Y^{t-1}}:~t=0,\ldots,n\}$ to maximize the finite-time horizon directed information defined by $$C^{FB}_{X^n \rightarrow Y^n} \triangleq \sup_{{\cal P}_{0,n}} I(X^n\rightarrow{Y^n}),~~~ I(X^n \rightarrow Y^n) =\sum_{t=0}^n{I}(X^t;Y_t|Y^{t-1})$$ for channe…
▽ More
We derive sequential necessary and sufficient conditions for any channel input conditional distribution ${\cal P}_{0,n}\triangleq\{P_{X_t|X^{t-1},Y^{t-1}}:~t=0,\ldots,n\}$ to maximize the finite-time horizon directed information defined by $$C^{FB}_{X^n \rightarrow Y^n} \triangleq \sup_{{\cal P}_{0,n}} I(X^n\rightarrow{Y^n}),~~~ I(X^n \rightarrow Y^n) =\sum_{t=0}^n{I}(X^t;Y_t|Y^{t-1})$$ for channel distributions $\{P_{Y_t|Y^{t-1},X_t}:~t=0,\ldots,n\}$ and $\{P_{Y_t|Y_{t-M}^{t-1},X_t}:~t=0,\ldots,n\}$, where $Y^t\triangleq\{Y_0,\ldots,Y_t\}$ and $X^t\triangleq\{X_0,\ldots,X_t\}$ are the channel input and output random processes, and $M$ is a finite nonnegative integer.
\noi We apply the necessary and sufficient conditions to application examples of time-varying channels with memory and we derive recursive closed form expressions of the optimal distributions, which maximize the finite-time horizon directed information. Further, we derive the feedback capacity from the asymptotic properties of the optimal distributions by investigating the limit $$C_{X^\infty \rightarrow Y^\infty}^{FB} \triangleq \lim_{n \longrightarrow \infty} \frac{1}{n+1} C_{X^n \rightarrow Y^n}^{FB}$$ without any á priori assumptions, such as, stationarity, ergodicity or irreducibility of the channel distribution. The necessary and sufficient conditions can be easily extended to a variety of channels with memory, beyond the ones considered in this paper.
△ Less
Submitted 18 April, 2016; v1 submitted 10 April, 2016;
originally announced April 2016.
-
Information Structures of Maximizing Distributions of Feedback Capacity for General Channels with Memory & Applications
Authors:
Charalambos D. Charalambous,
Christos K. Kourtellaris
Abstract:
For any class of channel conditional distributions, with finite memory dependence on channel input RVs $A^n {\stackrel{\triangle}{=}} \{A_i: i=0, \ldots, n\}$ or channel output RVs $B^n {\stackrel{\triangle}{=}} \{B_i: i=0, \ldots, n\}$ or both, we characterize the sets of channel input distributions, which maximize directed information defined by…
▽ More
For any class of channel conditional distributions, with finite memory dependence on channel input RVs $A^n {\stackrel{\triangle}{=}} \{A_i: i=0, \ldots, n\}$ or channel output RVs $B^n {\stackrel{\triangle}{=}} \{B_i: i=0, \ldots, n\}$ or both, we characterize the sets of channel input distributions, which maximize directed information defined by $ I(A^n \rightarrow B^n) {\stackrel{\triangle}{=}} \sum_{i=0}^n I(A^i;B_i|B^{i-1}) $ and we derive the corresponding expressions, called "characterizations of Finite Transmission Feedback Information (FTFI) capacity". The main theorems state that optimal channel input distributions occur in subsets ${\cal P}_{[0,n]}^{CI}\subseteq {\cal P}_{[0,n]} {\stackrel{\triangle}{=}}\big\{ {\bf P}_{A_i|A^{i-1}, B^{i-1}}: i=0, \ldots, n\big\}$, which satisfy conditional independence on past information. We derive similar characterizations, when general transmission cost constraints are imposed. Moreover, we also show that the structural properties apply to general nonlinear and linear autoregressive channel models defined by discrete-time recursions on general alphabet spaces, and driven by arbitrary distributed noise processes. We derive these structural properties by invoking stochastic optimal control theory and variational equalities of directed information, to identify tight upper bounds on $I(A^n \rightarrow B^n)$, which are achievable over subsets of conditional distributions ${\cal P}_{[0,n]}^{CI} \subseteq {\cal P}_{[0,n]}$, which satisfy conditional independence and they are specified by the dependence of channel distributions and transmission cost functions on inputs and output symbols. We apply the characterizations to recursive Multiple Input Multiple Output Gaussian Linear Channel Models with limited memory and we show a separation principle between the computation of the elements of the optimal strategies.
△ Less
Submitted 19 August, 2016; v1 submitted 4 April, 2016;
originally announced April 2016.
-
Capacity Achieving Distributions & Information Lossless Randomized Strategies for Feedback Channels with Memory: The LQG Theory of Directed Information-Part II
Authors:
Charalambos D. Charalambous,
Christos K. Kourtellaris,
Sergey Loyka
Abstract:
A methodology is developed to realized optimal channel input conditional distributions, which maximize the finite-time horizon directed information, for channels with memory and feedback, by information lossless randomized strategies. The methodology is applied to general Time-Varying Multiple Input Multiple Output (MIMO) Gaussian Linear Channel Models (G-LCMs) with memory, subject to average tran…
▽ More
A methodology is developed to realized optimal channel input conditional distributions, which maximize the finite-time horizon directed information, for channels with memory and feedback, by information lossless randomized strategies. The methodology is applied to general Time-Varying Multiple Input Multiple Output (MIMO) Gaussian Linear Channel Models (G-LCMs) with memory, subject to average transmission cost constraints of quadratic form. The realizations of optimal distributions by randomized strategies are shown to exhibit a decomposion into a deterministic part and a random part. The decomposition reveals the dual role of randomized strategies, to control the channel output process and to transmit new information over the channels. Moreover, a separation principle is shown between the computation of the optimal deterministic part and the random part of the randomized strategies. The dual role of randomized strategies generalizes the Linear-Quadratic-Gaussian (LQG) stochastic optimal control theory to directed information pay-offs. The characterizations of feedback capacity are obtained from the per unit time limits of finite-time horizon directed information, without imposing á priori assumptions, such as, stability of channel models or ergodicity of channel input and output processes. For time-invariant MIMO G-LCMs with memory, it is shown that whether feedback increases capacity, is directly related to the channel parameters and the transmission cost function, through the solutions of Riccati matrix equations, and moreover for unstable channels, feedback capacity is non-zero, provided the power exceeds a critical level.
△ Less
Submitted 4 April, 2016;
originally announced April 2016.
-
Information Structures for Feedback Capacity of Channels with Memory and Transmission Cost: Stochastic Optimal Control & Variational Equalities-Part I
Authors:
Christos K. Kourtellaris,
Charalambos D. Charalambous
Abstract:
The Finite Transmission Feedback Information (FTFI) capacity is characterized for any class of channel conditional distributions $\big\{{\bf P}_{B_i|B^{i-1}, A_i} :i=0, 1, \ldots, n\big\}$ and $\big\{ {\bf P}_{B_i|B_{i-M}^{i-1}, A_i} :i=0, 1, \ldots, n\big\}$, where $M$ is the memory of the channel, $B^n {\stackrel{\triangle}{=}} \{B_j: j=\ldots, 0,1, \ldots, n\}$ are the channel outputs and…
▽ More
The Finite Transmission Feedback Information (FTFI) capacity is characterized for any class of channel conditional distributions $\big\{{\bf P}_{B_i|B^{i-1}, A_i} :i=0, 1, \ldots, n\big\}$ and $\big\{ {\bf P}_{B_i|B_{i-M}^{i-1}, A_i} :i=0, 1, \ldots, n\big\}$, where $M$ is the memory of the channel, $B^n {\stackrel{\triangle}{=}} \{B_j: j=\ldots, 0,1, \ldots, n\}$ are the channel outputs and $A^n{\stackrel{\triangle}{=}} \{A_j: j=\ldots, 0,1, \ldots, n\}$ are the channel inputs. The characterizations of FTFI capacity, are obtained by first identifying the information structures of the optimal channel input conditional distributions ${\cal P}_{[0,n]} {\stackrel{\triangle}{=}} \big\{ {\bf P}_{A_i|A^{i-1}, B^{i-1}}: i=0, \ldots, n\big\}$, which maximize directed information. The main theorem states, for any channel with memory $M$, the optimal channel input conditional distributions occur in the subset satisfying conditional independence $\stackrel{\circ}{\cal P}_{[0,n]}{\stackrel{\triangle}{=}} \big\{ {\bf P}_{A_i|A^{i-1}, B^{i-1}}= {\bf P}_{A_i|B_{i-M}^{i-1}}: i=1, \ldots, n\big\}$, and the characterization of FTFI capacity is given by $C_{A^n \rightarrow B^n}^{FB, M} {\stackrel{\triangle}{=}} \sup_{ \stackrel{\circ}{\cal P}_{[0,n]} } \sum_{i=0}^n I(A_i; B_i|B_{i-M}^{i-1}) $. The methodology utilizes stochastic optimal control theory and a variational equality of directed information, to derive upper bounds on $I(A^n \rightarrow B^n)$, which are achievable over specific subsets of channel input conditional distributions ${\cal P}_{[0,n]}$, which are characterized by conditional independence. For any of the above classes of channel distributions and transmission cost functions, a direct analogy, in terms of conditional independence, of the characterizations of FTFI capacity and Shannon's capacity formulae of Memoryless Channels is identified.
△ Less
Submitted 18 August, 2016; v1 submitted 14 December, 2015;
originally announced December 2015.
-
Information Nonanticipative Rate Distortion Function and Its Applications
Authors:
Photios A. Stavrou,
Christos K. Kourtellaris,
C. D. Charalambous
Abstract:
This paper investigates applications of nonanticipative Rate Distortion Function (RDF) in a) zero-delay Joint Source-Channel Coding (JSCC) design based on average and excess distortion probability, b) in bounding the Optimal Performance Theoretically Attainable (OPTA) by noncausal and causal codes, and computing the Rate Loss (RL) of zero-delay and causal codes with respect to noncausal codes. The…
▽ More
This paper investigates applications of nonanticipative Rate Distortion Function (RDF) in a) zero-delay Joint Source-Channel Coding (JSCC) design based on average and excess distortion probability, b) in bounding the Optimal Performance Theoretically Attainable (OPTA) by noncausal and causal codes, and computing the Rate Loss (RL) of zero-delay and causal codes with respect to noncausal codes. These applications are described using two running examples, the Binary Symmetric Markov Source with parameter p, (BSMS(p)) and the multidimensional partially observed Gaussian-Markov source. For the multidimensional Gaussian-Markov source with square error distortion, the solution of the nonanticipative RDF is derived, its operational meaning using JSCC design via a noisy coding theorem is shown by providing the optimal encoding-decoding scheme over a vector Gaussian channel, and the RL of causal and zero-delay codes with respect to noncausal codes is computed.
For the BSMS(p) with Hamming distortion, the solution of the nonanticipative RDF is derived, the RL of causal codes with respect to noncausal codes is computed, and an uncoded noisy coding theorem based on excess distortion probability is shown. The information nonanticipative RDF is shown to be equivalent to the nonanticipatory epsilon-entropy, which corresponds to the classical RDF with an additional causality or nonanticipative condition imposed on the optimal reproduction conditional distribution.
△ Less
Submitted 18 November, 2015; v1 submitted 7 May, 2014;
originally announced May 2014.
-
Capacity of Binary State Symmetric Channel with and without Feedback and Transmission Cost
Authors:
Christos K. Kourtellaris,
Charalambos D. Charalambous
Abstract:
We consider a unit memory channel, called Binary State Symmetric Channel (BSSC), in which the channel state is the modulo2 addition of the current channel input and the previous channel output. We derive closed form expressions for the capacity and corresponding channel input distribution, of this BSSC with and without feedback and transmission cost. We also show that the capacity of the BSSC is n…
▽ More
We consider a unit memory channel, called Binary State Symmetric Channel (BSSC), in which the channel state is the modulo2 addition of the current channel input and the previous channel output. We derive closed form expressions for the capacity and corresponding channel input distribution, of this BSSC with and without feedback and transmission cost. We also show that the capacity of the BSSC is not increased by feedback, and it is achieved by a first order symmetric Markov process.
△ Less
Submitted 15 February, 2015; v1 submitted 31 January, 2014;
originally announced February 2014.
-
Applications of Information Nonanticipative Rate Distortion Function
Authors:
Photios A. Stavrou,
Christos K. Kourtellaris,
Charalambos D. Charalambous
Abstract:
The objective of this paper is to further investigate various applications of information Nonanticipative Rate Distortion Function (NRDF) by discussing two working examples, the Binary Symmetric Markov Source with parameter $p$ (BSMS($p$)) with Hamming distance distortion, and the multidimensional partially observed Gaussian-Markov source. For the BSMS($p$), we give the solution to the NRDF, and w…
▽ More
The objective of this paper is to further investigate various applications of information Nonanticipative Rate Distortion Function (NRDF) by discussing two working examples, the Binary Symmetric Markov Source with parameter $p$ (BSMS($p$)) with Hamming distance distortion, and the multidimensional partially observed Gaussian-Markov source. For the BSMS($p$), we give the solution to the NRDF, and we use it to compute the Rate Loss (RL) of causal codes with respect to noncausal codes. For the multidimensional Gaussian-Markov source, we give the solution to the NRDF, we show its operational meaning via joint source-channel matching over a vector of parallel Gaussian channels, and we compute the RL of causal and zero-delay codes with respect to noncausal codes.
△ Less
Submitted 29 April, 2014; v1 submitted 22 January, 2014;
originally announced January 2014.
-
Source-Channel Matching for Sources with Memory
Authors:
Christos Kourtellaris,
Charalambos D. Charalambous,
Photios A. Stavrou
Abstract:
In this paper we analyze the probabilistic matching of sources with memory to channels with memory so that symbol-by-symbol code with memory without anticipation are optimal, with respect to an average distortion and excess distortion probability. We show achievability of such a symbolby- symbol code with memory without anticipation, and we show matching for the Binary Symmetric Markov source (BSM…
▽ More
In this paper we analyze the probabilistic matching of sources with memory to channels with memory so that symbol-by-symbol code with memory without anticipation are optimal, with respect to an average distortion and excess distortion probability. We show achievability of such a symbolby- symbol code with memory without anticipation, and we show matching for the Binary Symmetric Markov source (BSMS(p)) over a first-order symmetric channel with a cost constraint.
△ Less
Submitted 25 March, 2014; v1 submitted 11 October, 2013;
originally announced October 2013.
-
Nonanticipative Rate Distortion Function for General Source-Channel Matching
Authors:
Christos Kourtellaris,
Charalambos D. Charalambous,
Photios A. Stavrou
Abstract:
In this paper we invoke a nonanticipative information Rate Distortion Function (RDF) for sources with memory, and we analyze its importance in probabilistic matching of the source to the channel so that transmission of a symbol-by-symbol code with memory without anticipation is optimal, with respect to an average distortion and excess distortion probability. We show achievability of the symbol-by-…
▽ More
In this paper we invoke a nonanticipative information Rate Distortion Function (RDF) for sources with memory, and we analyze its importance in probabilistic matching of the source to the channel so that transmission of a symbol-by-symbol code with memory without anticipation is optimal, with respect to an average distortion and excess distortion probability. We show achievability of the symbol-by-symbol code with memory without anticipation, and we evaluate the probabilistic performance of the code for a Markov source.
△ Less
Submitted 25 April, 2013; v1 submitted 24 April, 2013;
originally announced April 2013.
-
Optimal Nonstationary Reproduction Distribution for Nonanticipative RDF on Abstract Alphabets
Authors:
Photios A. Stavrou,
Charalambos D. Charalambous,
Christos K. Kourtellaris
Abstract:
In this paper we introduce a definition for nonanticipative Rate Distortion Function (RDF) on abstract alphabets, and we invoke weak convergence of probability measures to show various of its properties, such as, existence of the optimal reproduction conditional distribution, compactness of the fidelity set, lower semicontinuity of the RDF functional, etc. Further, we derive the closed form expres…
▽ More
In this paper we introduce a definition for nonanticipative Rate Distortion Function (RDF) on abstract alphabets, and we invoke weak convergence of probability measures to show various of its properties, such as, existence of the optimal reproduction conditional distribution, compactness of the fidelity set, lower semicontinuity of the RDF functional, etc. Further, we derive the closed form expression of the optimal nonstationary reproduction distribution. This expression is computed recursively backward in time. Throughout the paper we point out an operational meaning of the nonanticipative RDF by recalling the coding theorem derive in \cite{tatikonda2000}, and we state relations to Gorbunov-Pinsker's nonanticipatory $ε-$entropy \cite{gorbunov-pinsker}.
△ Less
Submitted 28 January, 2013;
originally announced January 2013.
-
Realizable Rate Distortion Function and Bayesian FIltering Theory
Authors:
Photios A. Stavrou,
Charalambos D. Charalambous,
Christos K. Kourtellaris
Abstract:
The relation between rate distortion function (RDF) and Bayesian filtering theory is discussed. The relation is established by imposing a causal or realizability constraint on the reconstruction conditional distribution of the RDF, leading to the definition of a causal RDF. Existence of the optimal reconstruction distribution of the causal RDF is shown using the topology of weak convergence of pro…
▽ More
The relation between rate distortion function (RDF) and Bayesian filtering theory is discussed. The relation is established by imposing a causal or realizability constraint on the reconstruction conditional distribution of the RDF, leading to the definition of a causal RDF. Existence of the optimal reconstruction distribution of the causal RDF is shown using the topology of weak convergence of probability measures. The optimal non-stationary causal reproduction conditional distribution of the causal RDF is derived in closed form; it is given by a set of recursive equations which are computed backward in time. The realization of causal RDF is described via the source-channel matching approach, while an example is briefly discussed to illustrate the concepts.
△ Less
Submitted 13 April, 2012;
originally announced April 2012.
-
Causal Rate Distortion Function on Abstract Alphabets: Optimal Reconstruction and Properties
Authors:
Photios A. Stavrou,
Charalambos D. Charalambous,
Christos K. Kourtellaris
Abstract:
A causal rate distortion function with a general fidelity criterion is formulated on abstract alphabets and a coding theorem is derived. Existence of the minimizing kernel is shown using the topology of weak convergence of probability measures. The optimal reconstruction kernel is derived, which is causal, and certain properties of the causal rate distortion function are presented.
A causal rate distortion function with a general fidelity criterion is formulated on abstract alphabets and a coding theorem is derived. Existence of the minimizing kernel is shown using the topology of weak convergence of probability measures. The optimal reconstruction kernel is derived, which is causal, and certain properties of the causal rate distortion function are presented.
△ Less
Submitted 4 February, 2012;
originally announced February 2012.
-
Causal Rate Distortion Function on Abstract Alphabets and Optimal Reconstruction Kernel
Authors:
Charalambos D. Charalambous,
Photios A. Stavrou,
Christos K. Kourtellaris
Abstract:
A Causal rate distortion function with a general fidelity criterion is formulated on abstract alphabets and the optimal reconstruction kernel is derived, which consists of a product of causal kernels. In the process, general abstract spaces are introduced to show existence of the minimizing kernel using weak*-convergence. Certain properties of the causal rate distortion function are presented.
A Causal rate distortion function with a general fidelity criterion is formulated on abstract alphabets and the optimal reconstruction kernel is derived, which consists of a product of causal kernels. In the process, general abstract spaces are introduced to show existence of the minimizing kernel using weak*-convergence. Certain properties of the causal rate distortion function are presented.
△ Less
Submitted 16 February, 2011;
originally announced February 2011.
