-
Joint Source-Channel Coding: Fundamentals and Recent Progress in Practical Designs
Authors:
Deniz Gündüz,
Michèle A. Wigger,
Tze-Yang Tung,
Ping Zhang,
Yong Xiao
Abstract:
Semantic- and task-oriented communication has emerged as a promising approach to reducing the latency and bandwidth requirements of next-generation mobile networks by transmitting only the most relevant information needed to complete a specific task at the receiver. This is particularly advantageous for machine-oriented communication of high data rate content, such as images and videos, where the…
▽ More
Semantic- and task-oriented communication has emerged as a promising approach to reducing the latency and bandwidth requirements of next-generation mobile networks by transmitting only the most relevant information needed to complete a specific task at the receiver. This is particularly advantageous for machine-oriented communication of high data rate content, such as images and videos, where the goal is rapid and accurate inference, rather than perfect signal reconstruction. While semantic- and task-oriented compression can be implemented in conventional communication systems, joint source-channel coding (JSCC) offers an alternative end-to-end approach by optimizing compression and channel coding together, or even directly mapping the source signal to the modulated waveform. Although all digital communication systems today rely on separation, thanks to its modularity, JSCC is known to achieve higher performance in finite blocklength scenarios, and to avoid cliff and the levelling-off effects in time-varying channel scenarios. This article provides an overview of the information theoretic foundations of JSCC, surveys practical JSCC designs over the decades, and discusses the reasons for their limited adoption in practical systems. We then examine the recent resurgence of JSCC, driven by the integration of deep learning techniques, particularly through DeepJSCC, highlighting its many surprising advantages in various scenarios. Finally, we discuss why it may be time to reconsider today's strictly separate architectures, and reintroduce JSCC to enable high-fidelity, low-latency communications in critical applications such as autonomous driving, drone surveillance, or wearable systems.
△ Less
Submitted 26 September, 2024;
originally announced September 2024.
-
Linear Sum Capacity for Gaussian Multiple Access Channels with Feedback
Authors:
Ehsan Ardestanizadeh,
Michele A. Wigger,
Young-Han Kim,
Tara Javidi
Abstract:
The capacity region of the N-sender Gaussian multiple access channel with feedback is not known in general. This paper studies the class of linear-feedback codes that includes (nonlinear) nonfeedback codes at one extreme and the linear-feedback codes by Schalkwijk and Kailath, Ozarow, and Kramer at the other extreme. The linear-feedback sum-capacity C_L(N,P) under symmetric power constraints P is…
▽ More
The capacity region of the N-sender Gaussian multiple access channel with feedback is not known in general. This paper studies the class of linear-feedback codes that includes (nonlinear) nonfeedback codes at one extreme and the linear-feedback codes by Schalkwijk and Kailath, Ozarow, and Kramer at the other extreme. The linear-feedback sum-capacity C_L(N,P) under symmetric power constraints P is characterized, the maximum sum-rate achieved by linear-feedback codes when each sender has the equal block power constraint P. In particular, it is shown that Kramer's code achieves this linear-feedback sum-capacity. The proof involves the dependence balance condition introduced by Hekstra and Willems and extended by Kramer and Gastpar, and the analysis of the resulting nonconvex optimization problem via a Lagrange dual formulation. Finally, an observation is presented based on the properties of the conditional maximal correlation---an extension of the Hirschfeld--Gebelein--Renyi maximal correlation---which reinforces the conjecture that Kramer's code achieves not only the linear-feedback sum-capacity, but also the sum-capacity itself (the maximum sum-rate achieved by arbitrary feedback codes).
△ Less
Submitted 2 June, 2011; v1 submitted 9 February, 2010;
originally announced February 2010.
-
On the Capacity of Free-Space Optical Intensity Channels
Authors:
Amos Lapidoth,
Stefan M. Moser,
Michele A. Wigger
Abstract:
New upper and lower bounds are presented on the capacity of the free-space optical intensity channel. This channel is characterized by inputs that are nonnegative (representing the transmitted optical intensity) and by outputs that are corrupted by additive white Gaussian noise (because in free space the disturbances arise from many independent sources). Due to battery and safety reasons the inp…
▽ More
New upper and lower bounds are presented on the capacity of the free-space optical intensity channel. This channel is characterized by inputs that are nonnegative (representing the transmitted optical intensity) and by outputs that are corrupted by additive white Gaussian noise (because in free space the disturbances arise from many independent sources). Due to battery and safety reasons the inputs are simultaneously constrained in both their average and peak power. For a fixed ratio of the average power to the peak power the difference between the upper and the lower bounds tends to zero as the average power tends to infinity, and the ratio of the upper and lower bounds tends to one as the average power tends to zero. The case where only an average-power constraint is imposed on the input is treated separately. In this case, the difference of the upper and lower bound tends to 0 as the average power tends to infinity, and their ratio tends to a constant as the power tends to zero.
△ Less
Submitted 10 March, 2009;
originally announced March 2009.
-
On the Gaussian MAC with Imperfect Feedback
Authors:
Amos Lapidoth,
Michele A. Wigger
Abstract:
New achievable rate regions are derived for the two-user additive white Gaussian multiple-access channel with noisy feedback. The regions exhibit the following two properties. Irrespective of the (finite) Gaussian feedback-noise variances, the regions include rate points that lie outside the no-feedback capacity region, and when the feedback-noise variances tend to 0 the regions converge to the p…
▽ More
New achievable rate regions are derived for the two-user additive white Gaussian multiple-access channel with noisy feedback. The regions exhibit the following two properties. Irrespective of the (finite) Gaussian feedback-noise variances, the regions include rate points that lie outside the no-feedback capacity region, and when the feedback-noise variances tend to 0 the regions converge to the perfect-feedback capacity region. The new achievable regions also apply to the partial-feedback setting where one of the transmitters has a noisy feedback link and the other transmitter has no feedback at all. Again, irrespective of the (finite) noise variance on the feedback link, the regions include rate points that lie outside the no-feedback capacity region. Moreover, in the case of perfect partial feedback, i.e., where the only feedback link is noise-free, for certain channel parameters the new regions include rate points that lie outside the Cover-Leung region. This answers in the negative the question posed by van der Meulen as to whether the Cover-Leung region equals the capacity region of the Gaussian multiple-access channel with perfect partial feedback. Finally, we propose new achievable regions also for a setting where the receiver is cognizant of the realizations of the noise sequences on the feedback links.
△ Less
Submitted 12 April, 2010; v1 submitted 5 February, 2009;
originally announced February 2009.
-
The pre-log of Gaussian broadcast with feedback can be two
Authors:
Michele A. Wigger,
Michael Gastpar
Abstract:
A generic intuition says that the pre-log, or multiplexing gain, cannot be larger than the minimum of the number of transmit and receive dimensions. This suggests that for the scalar broadcast channel, the pre-log cannot exceed one. By contrast, in this note, we show that when the noises are anti-correlated and feedback is present, then a pre-log of two can be attained. In other words, in this s…
▽ More
A generic intuition says that the pre-log, or multiplexing gain, cannot be larger than the minimum of the number of transmit and receive dimensions. This suggests that for the scalar broadcast channel, the pre-log cannot exceed one. By contrast, in this note, we show that when the noises are anti-correlated and feedback is present, then a pre-log of two can be attained. In other words, in this special case, in the limit of high SNR, the scalar Gaussian broadcast channel turns into two parallel AWGN channels. Achievability is established via a coding strategy due to Schalkwijk, Kailath, and Ozarow.
△ Less
Submitted 7 May, 2008;
originally announced May 2008.
-
On the Capacity of Free-Space Optical Intensity Channels
Authors:
Amos Lapidoth,
Stefan M. Moser,
Michele A. Wigger
Abstract:
New upper and lower bounds are presented on the capacity of the free-space optical intensity channel. This channel is characterized by inputs that are nonnegative (representing the transmitted optical intensity) and by outputs that are corrupted by additive white Gaussian noise (because in free space the disturbances arise from many independent sources). Due to battery and safety reasons the inp…
▽ More
New upper and lower bounds are presented on the capacity of the free-space optical intensity channel. This channel is characterized by inputs that are nonnegative (representing the transmitted optical intensity) and by outputs that are corrupted by additive white Gaussian noise (because in free space the disturbances arise from many independent sources). Due to battery and safety reasons the inputs are simultaneously constrained in both their average and peak power. For a fixed ratio of the average power to the peak power the difference between the upper and the lower bounds tends to zero as the average power tends to infinity, and the ratio of the upper and lower bounds tends to one as the average power tends to zero. The case where only an average-power constraint is imposed on the input is treated separately. In this case, the difference of the upper and lower bound tends to 0 as the average power tends to infinity, and their ratio tends to a constant as the power tends to zero.
△ Less
Submitted 5 May, 2008;
originally announced May 2008.
-
The Gaussian MAC with Conferencing Encoders
Authors:
Shraga I. Bross,
Amos Lapidoth,
Michele A. Wigger
Abstract:
We derive the capacity region of the Gaussian version of Willems's two-user MAC with conferencing encoders. This setting differs from the classical MAC in that, prior to each transmission block, the two transmitters can communicate with each other over noise-free bit-pipes of given capacities. The derivation requires a new technique for proving the optimality of Gaussian input distributions in c…
▽ More
We derive the capacity region of the Gaussian version of Willems's two-user MAC with conferencing encoders. This setting differs from the classical MAC in that, prior to each transmission block, the two transmitters can communicate with each other over noise-free bit-pipes of given capacities. The derivation requires a new technique for proving the optimality of Gaussian input distributions in certain mutual information maximizations under a Markov constraint. We also consider a Costa-type extension of the Gaussian MAC with conferencing encoders. In this extension, the channel can be described as a two-user MAC with Gaussian noise and Gaussian interference where the interference is known non-causally to the encoders but not to the decoder. We show that as in Costa's setting the interference sequence can be perfectly canceled, i.e., that the capacity region without interference can be achieved.
△ Less
Submitted 5 May, 2008;
originally announced May 2008.
-
On Cognitive Interference Networks
Authors:
Amos Lapidoth,
Shlomo Shamai,
Michele A. Wigger
Abstract:
We study the high-power asymptotic behavior of the sum-rate capacity of multi-user interference networks with an equal number of transmitters and receivers. We assume that each transmitter is cognizant of the message it wishes to convey to its corresponding receiver and also of the messages that a subset of the other transmitters wish to send. The receivers are assumed not to be able to cooperat…
▽ More
We study the high-power asymptotic behavior of the sum-rate capacity of multi-user interference networks with an equal number of transmitters and receivers. We assume that each transmitter is cognizant of the message it wishes to convey to its corresponding receiver and also of the messages that a subset of the other transmitters wish to send. The receivers are assumed not to be able to cooperate in any way so that they must base their decision on the signal they receive only. We focus on the network's pre-log, which is defined as the limiting ratio of the sum-rate capacity to half the logarithm of the transmitted power. We present both upper and lower bounds on the network's pre-log. The lower bounds are based on a linear partial-cancellation scheme which entails linearly transforming Gaussian codebooks so as to eliminate the interference in a subset of the receivers. Inter alias, the bounds give a complete characterization of the networks and side-information settings that result in a full pre-log, i.e., in a pre-log that is equal to the number of transmitters (and receivers) as well as a complete characterization of networks whose pre-log is equal to the full pre-log minus one. They also fully characterize networks where the full pre-log can only be achieved if each transmitter knows the messages of all users, i.e., when the side-information is "full".
△ Less
Submitted 6 July, 2007;
originally announced July 2007.
