Information Theory
See recent articles
Showing new listings for Friday, 16 May 2025
- [1] arXiv:2505.09644 [pdf, html, other]
-
Title: Joint Source-Channel Noise Adding with Adaptive Denoising for Diffusion-Based Semantic CommunicationsSubjects: Information Theory (cs.IT); Image and Video Processing (eess.IV)
Semantic communication (SemCom) aims to convey the intended meaning of messages rather than merely transmitting bits, thereby offering greater efficiency and robustness, particularly in resource-constrained or noisy environments. In this paper, we propose a novel framework which is referred to as joint source-channel noise adding with adaptive denoising (JSCNA-AD) for SemCom based on a diffusion model (DM). Unlike conventional encoder-decoder designs, our approach intentionally incorporates the channel noise during transmission, effectively transforming the harmful channel noise into a constructive component of the diffusion-based semantic reconstruction process. Besides, we introduce an attention-based adaptive denoising mechanism, in which transmitted images are divided into multiple regions, and the number of denoising steps is dynamically allocated based on the semantic importance of each region. This design effectively balances the reception quality and the inference latency by prioritizing the critical semantic information. Extensive experiments demonstrate that our method significantly outperforms existing SemCom schemes under various noise conditions, underscoring the potential of diffusion-based models in next-generation communication systems.
- [2] arXiv:2505.09940 [pdf, html, other]
-
Title: Low-Complexity Hybrid Beamforming for Multi-Cell mmWave Massive MIMO: A Primitive Kronecker Decomposition ApproachComments: 12 pages, 6 figures, 2 tables; accepted for publication in Signal ProcessingSubjects: Information Theory (cs.IT); Signal Processing (eess.SP)
To circumvent the high path loss of mmWave propagation and reduce the hardware cost of massive multiple-input multiple-output antenna systems, full-dimensional hybrid beamforming is critical in 5G and beyond wireless communications. Concerning an uplink multi-cell system with a large-scale uniform planar antenna array, this paper designs an efficient hybrid beamformer using primitive Kronecker decomposition and dynamic factor allocation, where the analog beamformer applies to null the inter-cell interference and simultaneously enhances the desired signals. In contrast, the digital beamformer mitigates the intra-cell interference using the minimum mean square error (MMSE) criterion. Then, due to the low accuracy of phase shifters inherent in the analog beamformer, a low-complexity hybrid beamformer is developed to slow its adjustment speed. Next, an optimality analysis from a subspace perspective is performed, and a sufficient condition for optimal antenna configuration is established. Finally, simulation results demonstrate that the achievable sum rate of the proposed beamformer approaches that of the optimal pure digital MMSE scheme, yet with much lower computational complexity and hardware cost.
- [3] arXiv:2505.09978 [pdf, html, other]
-
Title: Low-Complexity Decoding for Low-Rate Block Codes of Short Length Based on Concatenated Coding StructureMao-Chao Lin, Shih-Kai Lee, Pin Lin, Ching-Chang Lin, Chia-Chun Chen, Teng-Yuan Syu, Huang-Chang LeeSubjects: Information Theory (cs.IT); Signal Processing (eess.SP)
To decode a short linear block code, ordered statics decoding (OSD) and/or the $A^*$ decoding are usually considered. Either OSD or the $A^*$ decoding utilizes the magnitudes of the received symbols to establish the most reliable and independent positions (MRIP) frame. A restricted searched space can be employed to achieve near-optimum decoding with reduced decoding complexity. For a low-rate code with large minimum distance, the restricted search space is still very huge.
We propose to use concatenated coding to further restrict the search space by proposing an improved MRIP frame. The improved MRIP frame is founded according to magnitudes of log likelihood ratios (LLRs) obtained by the soft-in soft-out (SISO) decoder for the inner code.
We focus on the construction and decoding of several $(n,k)$ = (128,36) binary linear block codes based on concatenated coding. We use the (128,36) extended BCH (eBCH) code as a benchmark for comparison. Simulation shows that there exist constructed concatenated codes which are much more efficient than the (128,36) eBCH code. Some other codes of length 128 or close to 128 are also constructed to demonstrate the efficiency of the proposed scheme. - [4] arXiv:2505.10068 [pdf, html, other]
-
Title: The Schur product of evaluation codes and its application to CSS-T quantum codes and private information retrievalSubjects: Information Theory (cs.IT)
In this work, we study the componentwise (Schur) product of monomial-Cartesian codes by exploiting its correspondence with the Minkowski sum of their defining exponent sets. We show that $ J$-affine variety codes are well suited for such products, generalizing earlier results for cyclic, Reed-Muller, hyperbolic, and toric codes. Using this correspondence, we construct CSS-T quantum codes from weighted Reed-Muller codes and from binary subfield-subcodes of $ J$-affine variety codes, leading to codes with better parameters than previously known. Finally, we present Private Information Retrieval (PIR) constructions for multiple colluding servers based on hyperbolic codes and subfield-subcodes of $ J$-affine variety codes, and show that they outperform existing PIR schemes.
- [5] arXiv:2505.10184 [pdf, html, other]
-
Title: The Tangent Space AttackComments: 26 pages without references and appendix; 31 in totalSubjects: Information Theory (cs.IT); Cryptography and Security (cs.CR)
We propose a new method for retrieving the algebraic structure of a generic alternant code given an arbitrary generator matrix, provided certain conditions are met. We then discuss how this challenges the security of the McEliece cryptosystem instantiated with this family of codes. The central object of our work is the quadratic hull related to a linear code, defined as the intersection of all quadrics passing through the columns of a given generator or parity-check matrix, where the columns are considered as points in the affine or projective space. The geometric properties of this object reveal important information about the internal algebraic structure of the code. This is particularly evident in the case of generalized Reed-Solomon codes, whose quadratic hull is deeply linked to a well-known algebraic variety called the rational normal curve. By utilizing the concept of Weil restriction of affine varieties, we demonstrate that the quadratic hull of a generic dual alternant code inherits many interesting features from the rational normal curve, on account of the fact that alternant codes are subfield-subcodes of generalized Reed-Solomon codes. If the rate of the generic alternant code is sufficiently high, this allows us to construct a polynomial-time algorithm for retrieving the underlying generalized Reed-Solomon code from which the alternant code is defined, which leads to an efficient key-recovery attack against the McEliece cryptosystem when instantiated with this class of codes. Finally, we discuss the generalization of this approach to Algebraic-Geometry codes and Goppa codes.
New submissions (showing 5 of 5 entries)
- [6] arXiv:2505.09647 (cross-list from cs.DS) [pdf, html, other]
-
Title: On Unbiased Low-Rank Approximation with Minimum DistortionSubjects: Data Structures and Algorithms (cs.DS); Information Theory (cs.IT); Machine Learning (cs.LG); Probability (math.PR); Statistics Theory (math.ST)
We describe an algorithm for sampling a low-rank random matrix $Q$ that best approximates a fixed target matrix $P\in\mathbb{C}^{n\times m}$ in the following sense: $Q$ is unbiased, i.e., $\mathbb{E}[Q] = P$; $\mathsf{rank}(Q)\leq r$; and $Q$ minimizes the expected Frobenius norm error $\mathbb{E}\|P-Q\|_F^2$. Our algorithm mirrors the solution to the efficient unbiased sparsification problem for vectors, except applied to the singular components of the matrix $P$. Optimality is proven by showing that our algorithm matches the error from an existing lower bound.
- [7] arXiv:2505.09851 (cross-list from cs.LG) [pdf, other]
-
Title: ZENN: A Thermodynamics-Inspired Computational Framework for Heterogeneous Data-Driven ModelingComments: 9 pages, 4 figuresSubjects: Machine Learning (cs.LG); Information Theory (cs.IT)
Traditional entropy-based methods - such as cross-entropy loss in classification problems - have long been essential tools for quantifying uncertainty and disorder in data and developing artificial intelligence algorithms. However, the rapid growth of data across various domains has introduced new challenges, particularly the integration of heterogeneous datasets with intrinsic disparities. In this paper, we extend zentropy theory into the data science domain by introducing intrinsic entropy, enabling more effective learning from heterogeneous data sources. We propose a zentropy-enhanced neural network (ZENN) that simultaneously learns both energy and intrinsic entropy components, capturing the underlying structure of multi-source data. To support this, we redesign the neural network architecture to better reflect the intrinsic properties and variability inherent in diverse datasets. We demonstrate the effectiveness of ZENN on classification tasks and energy landscape reconstructions, showing its superior generalization capabilities and robustness-particularly in predicting high-order derivatives. As a practical application, we employ ZENN to reconstruct the Helmholtz energy landscape of Fe3Pt using data generated from DFT and capture key material behaviors, including negative thermal expansion and the critical point in the temperature-pressure space. Overall, our study introduces a novel approach for data-driven machine learning grounded in zentropy theory, highlighting ZENN as a versatile and robust deep learning framework for scientific problems involving complex, heterogeneous datasets.
- [8] arXiv:2505.09893 (cross-list from math.CO) [pdf, html, other]
-
Title: On null completely regular codes in Manhattan metricSubjects: Combinatorics (math.CO); Information Theory (cs.IT)
We investigate the class of completely regular codes in graphs with a distance partition C_0,..., C_\rho, where each set C_i, for 0<=i<=r-1, is an independent set. This work focuses on the existence problem for such codes in the n-dimensional infinite grid. We demonstrate that several parameter families of such codes necessarily arise from binary or ternary Hamming graphs or do not exist. Furthermore, employing binary linear programming techniques, we explore completely regular codes in infinite grids of dimensions 3 and 4 for the cases r=1 and r=2.
Cross submissions (showing 3 of 3 entries)
- [9] arXiv:2501.07279 (replaced) [pdf, html, other]
-
Title: Toward Universal Decoding of Binary Linear Block Codes via Enhanced Polar TransformationsSubjects: Information Theory (cs.IT)
Binary linear block codes (BLBCs) are essential to modern communication, but their diverse structures often require tailor-made decoders, increasing complexity. This work introduces enhanced polar decoding ($\mathsf{PD}^+$), a universal soft decoding algorithm that transforms any BLBC into a polar-like code compatible with efficient polar code decoders such as successive cancellation list (SCL) decoding. Key innovations in $\mathsf{PD}^+$ include pruning polar kernels, shortening codes, and leveraging a simulated annealing algorithm to optimize transformations. These enable $\mathsf{PD}^+$ to achieve competitive or superior performance to state-of-the-art algorithms like OSD and GRAND across various codes, including extended BCH, extended Golay, and binary quadratic residue codes, with significantly lower complexity. Moreover, $\mathsf{PD}^+$ is designed to be forward-compatible with advancements in polar code decoding techniques and AI-driven search methods, making it a robust and versatile solution for universal BLBC decoding in both present and future systems.
- [10] arXiv:2503.09977 (replaced) [pdf, html, other]
-
Title: Quadratic Transform for Fractional Programming in Signal Processing and Machine LearningComments: 20 pagesJournal-ref: IEEE Signal Processing Magazine 2025Subjects: Information Theory (cs.IT)
Fractional programming (FP) is a branch of mathematical optimization that deals with the optimization of ratios. It is an invaluable tool for signal processing and machine learning, because many key metrics in these fields are fractionally structured, e.g., the signal-to-interference-plus-noise ratio (SINR) in wireless communications, the Cramér-Rao bound (CRB) in radar sensing, the normalized cut in graph clustering, and the margin in support vector machine (SVM). This article provides a comprehensive review of both the theory and applications of a recently developed FP technique known as the quadratic transform, which can be applied to a wide variety of FP problems, including both the minimization and the maximization of the sum of functions of ratios as well as matrix-ratio problems.
- [11] arXiv:2505.05676 (replaced) [pdf, html, other]
-
Title: An Efficient Transport-Based Dissimilarity Measure for Time Series Classification under Warping DistortionsAkram Aldroubi, Rocío Díaz Martín, Ivan Medri, Kristofor E. Pas, Gustavo K. Rohde, Abu Hasnat Mohammad RubaiyatSubjects: Information Theory (cs.IT); Machine Learning (stat.ML)
Time Series Classification (TSC) is an important problem with numerous applications in science and technology. Dissimilarity-based approaches, such as Dynamic Time Warping (DTW), are classical methods for distinguishing time series when time deformations are confounding information. In this paper, starting from a deformation-based model for signal classes we define a problem statement for time series classification problem. We show that, under theoretically ideal conditions, a continuous version of classic 1NN-DTW method can solve the stated problem, even when only one training sample is available. In addition, we propose an alternative dissimilarity measure based on Optimal Transport and show that it can also solve the aforementioned problem statement at a significantly reduced computational cost. Finally, we demonstrate the application of the newly proposed approach in simulated and real time series classification data, showing the efficacy of the method.
- [12] arXiv:2311.08168 (replaced) [pdf, html, other]
-
Title: Time-Uniform Confidence Spheres for Means of Random VectorsComments: 52 pages; 3 figures. Published in Transactions on Machine Learning ResearchSubjects: Statistics Theory (math.ST); Information Theory (cs.IT); Methodology (stat.ME); Machine Learning (stat.ML)
We study sequential mean estimation in $\mathbb{R}^d$. In particular, we derive time-uniform confidence spheres -- confidence sphere sequences (CSSs) -- which contain the mean of random vectors with high probability simultaneously across all sample sizes. Our results include a dimension-free CSS for log-concave random vectors, a dimension-free CSS for sub-Gaussian random vectors, and CSSs for sub-$\psi$ random vectors (which includes sub-gamma, sub-Poisson, and sub-exponential distributions). Many of our results are optimal. For sub-Gaussian distributions we also provide a CSS which tracks a time-varying mean, generalizing Robbins' mixture approach to the multivariate setting. Finally, we provide several CSSs for heavy-tailed random vectors (two moments only). Our bounds hold under a martingale assumption on the mean and do not require that the observations be iid. Our work is based on PAC-Bayesian theory and inspired by an approach of Catoni and Giulini.
- [13] arXiv:2501.13018 (replaced) [pdf, html, other]
-
Title: Multi-Objective Hyperparameter Selection via Hypothesis Testing on Reliability GraphsSubjects: Machine Learning (cs.LG); Information Theory (cs.IT)
The selection of hyperparameters, such as prompt templates in large language models (LLMs), must often strike a balance between reliability and cost. In many cases, structural relationships between the expected reliability levels of the hyperparameters can be inferred from prior information and held-out data -- e.g., longer prompt templates may be more detailed and thus more reliable. However, existing hyperparameter selection methods either do not provide formal reliability guarantees or are unable to incorporate structured knowledge in the hyperparameter space. This paper introduces reliability graph-based Pareto testing (RG-PT), a novel multi-objective hyperparameter selection framework that maintains formal reliability guarantees in terms of false discovery rate (FDR), while accounting for known relationships among hyperparameters via a directed acyclic graph. Edges in the graph reflect expected reliability and cost trade-offs among hyperparameters, which are inferred via the Bradley-Terry (BT) ranking model from prior information and held-out data. Experimental evaluations demonstrate that RG-PT significantly outperforms existing methods such as learn-then-test (LTT) and Pareto testing (PT) through a more efficient exploration of the hyperparameter space.
- [14] arXiv:2504.17185 (replaced) [pdf, other]
-
Title: Compact Lattice-Coded (Multi-Recipient) Kyber without CLT Independence AssumptionComments: 8 Tables, 3 FiguresSubjects: Cryptography and Security (cs.CR); Information Theory (cs.IT)
This work presents a joint design of encoding and encryption procedures for public key encryptions (PKEs) and key encapsulation mechanism (KEMs) such as Kyber, without relying on the assumption of independent decoding noise components, achieving reductions in both communication overhead (CER) and decryption failure rate (DFR). Our design features two techniques: ciphertext packing and lattice packing. First, we extend the Peikert-Vaikuntanathan-Waters (PVW) method to Kyber: $\ell$ plaintexts are packed into a single ciphertext. This scheme is referred to as P$_\ell$-Kyber. We prove that the P$_\ell$-Kyber is IND-CCA secure under the M-LWE hardness assumption. We show that the decryption decoding noise entries across the $\ell$ plaintexts (also known as layers) are mutually independent. Second, we propose a cross-layer lattice encoding scheme for the P$_\ell$-Kyber, where every $\ell$ cross-layer information symbols are encoded to a lattice point. This way we obtain a \emph{coded} P$_\ell$-Kyber, where the decoding noise entries for each lattice point are mutually independent. Therefore, the DFR analysis does not require the assumption of independence among the decryption decoding noise entries. Both DFR and CER are greatly decreased thanks to ciphertext packing and lattice packing. We demonstrate that with $\ell=24$ and Leech lattice encoder, the proposed coded P$_\ell$-KYBER1024 achieves DFR $<2^{-281}$ and CER $ = 4.6$, i.e., a decrease of CER by $90\%$ compared to KYBER1024.