Showing 1–50 of 115 results for author: Kountouris, M

Variational Inference for Quantum HyperNetworks

Authors: Luca Nepote, Alix Lhéritier, Nicolas Bondoux, Marios Kountouris, Maurizio Filippone

Abstract: Binary Neural Networks (BiNNs), which employ single-bit precision weights, have emerged as a promising solution to reduce memory usage and power consumption while maintaining competitive performance in large-scale systems. However, training BiNNs remains a significant challenge due to the limitations of conventional training algorithms. Quantum HyperNetworks offer a novel paradigm for enhancing th… ▽ More Binary Neural Networks (BiNNs), which employ single-bit precision weights, have emerged as a promising solution to reduce memory usage and power consumption while maintaining competitive performance in large-scale systems. However, training BiNNs remains a significant challenge due to the limitations of conventional training algorithms. Quantum HyperNetworks offer a novel paradigm for enhancing the optimization of BiNN by leveraging quantum computing. Specifically, a Variational Quantum Algorithm is employed to generate binary weights through quantum circuit measurements, while key quantum phenomena such as superposition and entanglement facilitate the exploration of a broader solution space. In this work, we establish a connection between this approach and Bayesian inference by deriving the Evidence Lower Bound (ELBO), when direct access to the output distribution is available (i.e., in simulations), and introducing a surrogate ELBO based on the Maximum Mean Discrepancy (MMD) metric for scenarios involving implicit distributions, as commonly encountered in practice. Our experimental results demonstrate that the proposed methods outperform standard Maximum Likelihood Estimation (MLE), improving trainability and generalization. △ Less

Submitted 6 June, 2025; originally announced June 2025.

Comments: This work has been accepted for publication in 2025 International Joint Conference on Neural Networks (IJCNN 2025) and will be published on IEEE Xplore

MSC Class: 68Q12

Optimization for Semantic-Aware Resource Allocation under CPT-based Utilities

Authors: Symeon Vaidanis, Photios A. Stavrou, Marios Kountouris

Abstract: The problem of resource allocation in goal-oriented semantic communication with semantic-aware utilities and subjective risk perception is studied here. By linking information importance to risk aversion, we model agent behavior using Cumulative Prospect Theory (CPT), which incorporates risk-sensitive utility functions and nonlinear transformations of distributions, reflecting subjective perceptio… ▽ More The problem of resource allocation in goal-oriented semantic communication with semantic-aware utilities and subjective risk perception is studied here. By linking information importance to risk aversion, we model agent behavior using Cumulative Prospect Theory (CPT), which incorporates risk-sensitive utility functions and nonlinear transformations of distributions, reflecting subjective perceptions of gains and losses. The objective is to maximize the aggregate utility across multiple CPT-modeled agents, which leads to a nonconvex, nonsmooth optimization problem. To efficiently solve this challenging problem, we propose a new algorithmic framework that combines successive convex approximation (SCA) with the projected subgradient method and Lagrangian relaxation, Our approach enables tractable optimization while preserving solution quality, offering both theoretical rigor and practical effectiveness in semantics-aware resource allocation. △ Less

Submitted 5 June, 2025; originally announced June 2025.

Comments: This work has been accepted for publication in IEEE SPAWC 2025. The final published version will be available via IEEE Xplore

Goal-Oriented Semantic Resource Allocation with Cumulative Prospect Theoretic Agents

Authors: Symeon Vaidanis, Photios A. Stavrou, Marios Kountouris

Abstract: We introduce a resource allocation framework for goal-oriented semantic networks, where participating agents assess system quality through subjective (e.g., context-dependent) perceptions. To accommodate this, our model accounts for agents whose preferences deviate from traditional expected utility theory (EUT), specifically incorporating cumulative prospect theory (CPT) preferences. We develop a… ▽ More We introduce a resource allocation framework for goal-oriented semantic networks, where participating agents assess system quality through subjective (e.g., context-dependent) perceptions. To accommodate this, our model accounts for agents whose preferences deviate from traditional expected utility theory (EUT), specifically incorporating cumulative prospect theory (CPT) preferences. We develop a comprehensive analytical framework that captures human-centric aspects of decision-making and risky choices under uncertainty, such as risk perception, loss aversion, and perceptual distortions in probability metrics. By identifying essential modifications in traditional resource allocation design principles required for agents with CPT preferences, we showcase the framework's relevance through its application to the problem of power allocation in multi-channel wireless communication systems. △ Less

Submitted 5 June, 2025; originally announced June 2025.

Comments: This work has been accepted for publication in IEEE ICC 2025. The final published version will be available via IEEE Xplore

A Conformal Predictive Measure for Assessing Catastrophic Forgetting

Authors: Ioannis Pitsiorlas, Nour Jamoussi, Marios Kountouris

Abstract: This work introduces a novel methodology for assessing catastrophic forgetting (CF) in continual learning. We propose a new conformal prediction (CP)-based metric, termed the Conformal Prediction Confidence Factor (CPCF), to quantify and evaluate CF effectively. Our framework leverages adaptive CP to estimate forgetting by monitoring the model's confidence on previously learned tasks. This approac… ▽ More This work introduces a novel methodology for assessing catastrophic forgetting (CF) in continual learning. We propose a new conformal prediction (CP)-based metric, termed the Conformal Prediction Confidence Factor (CPCF), to quantify and evaluate CF effectively. Our framework leverages adaptive CP to estimate forgetting by monitoring the model's confidence on previously learned tasks. This approach provides a dynamic and practical solution for monitoring and measuring CF of previous tasks as new ones are introduced, offering greater suitability for real-world applications. Experimental results on four benchmark datasets demonstrate a strong correlation between CPCF and the accuracy of previous tasks, validating the reliability and interpretability of the proposed metric. Our results highlight the potential of CPCF as a robust and effective tool for assessing and understanding CF in dynamic learning environments. △ Less

Submitted 15 May, 2025; originally announced May 2025.

Pull-Based Query Scheduling for Goal-Oriented Semantic Communication

Authors: Pouya Agheli, Nikolaos Pappas, Marios Kountouris

Abstract: This paper addresses query scheduling for goal-oriented semantic communication in pull-based status update systems. We consider a system where multiple sensing agents (SAs) observe a source characterized by various attributes and provide updates to multiple actuation agents (AAs), which act upon the received information to fulfill their heterogeneous goals at the endpoint. A hub serves as an inter… ▽ More This paper addresses query scheduling for goal-oriented semantic communication in pull-based status update systems. We consider a system where multiple sensing agents (SAs) observe a source characterized by various attributes and provide updates to multiple actuation agents (AAs), which act upon the received information to fulfill their heterogeneous goals at the endpoint. A hub serves as an intermediary, querying the SAs for updates on observed attributes and maintaining a knowledge base, which is then broadcast to the AAs. The AAs leverage the knowledge to perform their actions effectively. To quantify the semantic value of updates, we introduce a grade of effectiveness (GoE) metric. Furthermore, we integrate cumulative perspective theory (CPT) into the long-term effectiveness analysis to account for risk awareness and loss aversion in the system. Leveraging this framework, we compute effect-aware scheduling policies aimed at maximizing the expected discounted sum of CPT-based total GoE provided by the transmitted updates while complying with a given query cost constraint. To achieve this, we propose a model-based solution based on dynamic programming and model-free solutions employing state-of-the-art deep reinforcement learning (DRL) algorithms. Our findings demonstrate that effect-aware scheduling significantly enhances the effectiveness of communicated updates compared to benchmark scheduling methods, particularly in settings with stringent cost constraints where optimal query scheduling is vital for system performance and overall effectiveness. △ Less

Submitted 9 March, 2025; originally announced March 2025.

Comments: Submitted for possible publication

Large-Scale AI in Telecom: Charting the Roadmap for Innovation, Scalability, and Enhanced Digital Experiences

Authors: Adnan Shahid, Adrian Kliks, Ahmed Al-Tahmeesschi, Ahmed Elbakary, Alexandros Nikou, Ali Maatouk, Ali Mokh, Amirreza Kazemi, Antonio De Domenico, Athanasios Karapantelakis, Bo Cheng, Bo Yang, Bohao Wang, Carlo Fischione, Chao Zhang, Chaouki Ben Issaid, Chau Yuen, Chenghui Peng, Chongwen Huang, Christina Chaccour, Christo Kurisummoottil Thomas, Dheeraj Sharma, Dimitris Kalogiros, Dusit Niyato, Eli De Poorter , et al. (110 additional authors not shown)

Abstract: This white paper discusses the role of large-scale AI in the telecommunications industry, with a specific focus on the potential of generative AI to revolutionize network functions and user experiences, especially in the context of 6G systems. It highlights the development and deployment of Large Telecom Models (LTMs), which are tailored AI models designed to address the complex challenges faced b… ▽ More This white paper discusses the role of large-scale AI in the telecommunications industry, with a specific focus on the potential of generative AI to revolutionize network functions and user experiences, especially in the context of 6G systems. It highlights the development and deployment of Large Telecom Models (LTMs), which are tailored AI models designed to address the complex challenges faced by modern telecom networks. The paper covers a wide range of topics, from the architecture and deployment strategies of LTMs to their applications in network management, resource allocation, and optimization. It also explores the regulatory, ethical, and standardization considerations for LTMs, offering insights into their future integration into telecom infrastructure. The goal is to provide a comprehensive roadmap for the adoption of LTMs to enhance scalability, performance, and user-centric innovation in telecom networks. △ Less

Submitted 6 March, 2025; originally announced March 2025.

CSI-Free Low-Complexity Remote State Estimation over Wireless MIMO Fading Channels using Semantic Analog Aggregation

Authors: Minjie Tang, Photios A. Stavrou, Marios Kountouris

Abstract: In this work, we investigate low-complexity remote system state estimation over wireless multiple-input-multiple-output (MIMO) channels without requiring prior knowledge of channel state information (CSI). We start by reviewing the conventional Kalman filtering-based state estimation algorithm, which typically relies on perfect CSI and incurs considerable computational complexity. To overcome the… ▽ More In this work, we investigate low-complexity remote system state estimation over wireless multiple-input-multiple-output (MIMO) channels without requiring prior knowledge of channel state information (CSI). We start by reviewing the conventional Kalman filtering-based state estimation algorithm, which typically relies on perfect CSI and incurs considerable computational complexity. To overcome the need for CSI, we introduce a novel semantic aggregation method, in which sensors transmit semantic measurement discrepancies to the remote state estimator through analog aggregation. To further reduce computational complexity, we introduce a constant-gain-based filtering algorithm that can be optimized offline using the constrained stochastic successive convex approximation (CSSCA) method. We derive a closed-form sufficient condition for the estimation stability of our proposed scheme via Lyapunov drift analysis. Numerical results showcase significant performance gains using the proposed scheme compared to several widely used methods. △ Less

Submitted 24 January, 2025; originally announced January 2025.

On the Rate-Distortion-Perception Function for Gaussian Processes

Authors: Giuseppe Serra, Photios A. Stavrou, Marios Kountouris

Abstract: In this paper, we investigate the rate-distortion-perception function (RDPF) of a source modeled by a Gaussian Process (GP) on a measure space $Ω$ under mean squared error (MSE) distortion and squared Wasserstein-2 perception metrics. First, we show that the optimal reconstruction process is itself a GP, characterized by a covariance operator sharing the same set of eigenvectors of the source cova… ▽ More In this paper, we investigate the rate-distortion-perception function (RDPF) of a source modeled by a Gaussian Process (GP) on a measure space $Ω$ under mean squared error (MSE) distortion and squared Wasserstein-2 perception metrics. First, we show that the optimal reconstruction process is itself a GP, characterized by a covariance operator sharing the same set of eigenvectors of the source covariance operator. Similarly to the classical rate-distortion function, this allows us to formulate the RDPF problem in terms of the Karhunen-Loève transform coefficients of the involved GPs. Leveraging the similarities with the finite-dimensional Gaussian RDPF, we formulate an analytical tight upper bound for the RDPF for GPs, which recovers the optimal solution in the "perfect realism" regime. Lastly, in the case where the source is a stationary GP and $Ω$ is the interval $[0, T]$ equipped with the Lebesgue measure, we derive an upper bound on the rate and the distortion for a fixed perceptual level and $T \to \infty$ as a function of the spectral density of the source process. △ Less

Submitted 10 January, 2025; originally announced January 2025.

Comments: This paper has been submitted for publication

So Timely, Yet So Stale: The Impact of Clock Drift in Real-Time Systems

Authors: Mehrdad Salimnejad, Nikolaos Pappas, Marios Kountouris

Abstract: In this paper, we address the problem of timely delivery of status update packets in a real-time communication system, where a transmitter sends status updates generated by a source to a receiver over an unreliable channel. The timestamps of transmitted and received packets are measured using separate clocks located at the transmitter and receiver, respectively. To account for possible clock drift… ▽ More In this paper, we address the problem of timely delivery of status update packets in a real-time communication system, where a transmitter sends status updates generated by a source to a receiver over an unreliable channel. The timestamps of transmitted and received packets are measured using separate clocks located at the transmitter and receiver, respectively. To account for possible clock drift between these two clocks, we consider both deterministic and probabilistic drift scenarios. We analyze the system's performance regarding the Age of Information (AoI) and derive closed-form expressions for the distribution and the average AoI under both clock drift models. Additionally, we explore the impact of key system parameters on the average AoI through analytical and numerical results. △ Less

Submitted 31 December, 2024; originally announced January 2025.

Information-Geometric Barycenters for Bayesian Federated Learning

Authors: Nour Jamoussi, Giuseppe Serra, Photios A. Stavrou, Marios Kountouris

Abstract: Federated learning (FL) is a widely used and impactful distributed optimization framework that achieves consensus through averaging locally trained models. While effective, this approach may not align well with Bayesian inference, where the model space has the structure of a distribution space. Taking an information-geometric perspective, we reinterpret FL aggregation as the problem of finding the… ▽ More Federated learning (FL) is a widely used and impactful distributed optimization framework that achieves consensus through averaging locally trained models. While effective, this approach may not align well with Bayesian inference, where the model space has the structure of a distribution space. Taking an information-geometric perspective, we reinterpret FL aggregation as the problem of finding the barycenter of local posteriors using a prespecified divergence metric, minimizing the average discrepancy across clients. This perspective provides a unifying framework that generalizes many existing methods and offers crisp insights into their theoretical underpinnings. We then propose BA-BFL, an algorithm that retains the convergence properties of Federated Averaging in non-convex settings. In non-independent and identically distributed scenarios, we conduct extensive comparisons with statistical aggregation techniques, showing that BA-BFL achieves performance comparable to state-of-the-art methods while offering a geometric interpretation of the aggregation phase. Additionally, we extend our analysis to Hybrid Bayesian Deep Learning, exploring the impact of Bayesian layers on uncertainty quantification and model calibration. △ Less

Submitted 7 May, 2025; v1 submitted 16 December, 2024; originally announced December 2024.

Optimizing Version Innovation Age for Monitoring Markovian Source in Energy-Harvesting Systems

Authors: Mehrdad Salimnejad, Anthony Ephremides, Marios Kountouris, Nikolaos Pappas

Abstract: We study the real-time remote tracking of a two-state Markov process by an energy harvesting source. The source decides whether to transmit over an unreliable channel based on the state. We formulate this scenario as a Markov decision process (MDP) to determine the optimal transmission policy that minimizes the average Version Innovation Age (VIA) as a performance metric. We demonstrate that the o… ▽ More We study the real-time remote tracking of a two-state Markov process by an energy harvesting source. The source decides whether to transmit over an unreliable channel based on the state. We formulate this scenario as a Markov decision process (MDP) to determine the optimal transmission policy that minimizes the average Version Innovation Age (VIA) as a performance metric. We demonstrate that the optimal transmission policy is threshold-based, determined by the battery level, source state, and VIA value. We numerically verify the analytical structure of the optimal policy and compare the performance of our proposed policy against two baseline policies across various system parameters, establishing the superior performance of our approach. △ Less

Submitted 15 October, 2024; originally announced October 2024.

Trustworthy Intrusion Detection: Confidence Estimation Using Latent Space

Authors: Ioannis Pitsiorlas, George Arvanitakis, Marios Kountouris

Abstract: This work introduces a novel method for enhancing confidence in anomaly detection in Intrusion Detection Systems (IDS) through the use of a Variational Autoencoder (VAE) architecture. By developing a confidence metric derived from latent space representations, we aim to improve the reliability of IDS predictions against cyberattacks. Applied to the NSL-KDD dataset, our approach focuses on binary c… ▽ More This work introduces a novel method for enhancing confidence in anomaly detection in Intrusion Detection Systems (IDS) through the use of a Variational Autoencoder (VAE) architecture. By developing a confidence metric derived from latent space representations, we aim to improve the reliability of IDS predictions against cyberattacks. Applied to the NSL-KDD dataset, our approach focuses on binary classification tasks to effectively distinguish between normal and malicious network activities. The methodology demonstrates a significant enhancement in anomaly detection, evidenced by a notable correlation of 0.45 between the reconstruction error and the proposed metric. Our findings highlight the potential of employing VAEs for more accurate and trustworthy anomaly detection in network security. △ Less

Submitted 19 September, 2024; originally announced September 2024.

Comments: 7 pages

Alternating Minimization Schemes for Computing Rate-Distortion-Perception Functions with $f$-Divergence Perception Constraints

Authors: Giuseppe Serra, Photios A. Stavrou, Marios Kountouris

Abstract: We study the computation of the rate-distortion-perception function (RDPF) for discrete memoryless sources subject to a single-letter average distortion constraint and a perception constraint that belongs to the family of $f$-divergences. In this setting, the RDPF forms a convex programming problem for which we characterize the optimal parametric solutions. We employ the developed solutions in an… ▽ More We study the computation of the rate-distortion-perception function (RDPF) for discrete memoryless sources subject to a single-letter average distortion constraint and a perception constraint that belongs to the family of $f$-divergences. In this setting, the RDPF forms a convex programming problem for which we characterize the optimal parametric solutions. We employ the developed solutions in an alternating minimization scheme, namely Optimal Alternating Minimization (OAM), for which we provide convergence guarantees. Nevertheless, the OAM scheme does not lead to a direct implementation of a generalized Blahut-Arimoto (BA) type of algorithm due to the presence of implicit equations in the structure of the iteration. To overcome this difficulty, we propose two alternative minimization approaches whose applicability depends on the smoothness of the used perception metric: a Newton-based Alternating Minimization (NAM) scheme, relying on Newton's root-finding method for the approximation of the optimal iteration solution, and a Relaxed Alternating Minimization (RAM) scheme, based on a relaxation of the OAM iterates. Both schemes are shown, via the derivation of necessary and sufficient conditions, to guarantee convergence to a globally optimal solution. We also provide sufficient conditions on the distortion and the perception constraints which guarantee that the proposed algorithms converge exponentially fast in the number of iteration steps. We corroborate our theoretical results with numerical simulations and draw connections with existing results. △ Less

Submitted 27 August, 2024; originally announced August 2024.

Comments: This work has been submitted for possible publication

Integrated Push-and-Pull Update Model for Goal-Oriented Effective Communication

Authors: Pouya Agheli, Nikolaos Pappas, Petar Popovski, Marios Kountouris

Abstract: This paper studies decision-making for goal-oriented effective communication. We consider an end-to-end status update system where a sensing agent (SA) observes a source, generates and transmits updates to an actuation agent (AA), while the AA takes actions to accomplish a goal at the endpoint. We integrate the push- and pull-based update communication models to obtain a push-and-pull model, which… ▽ More This paper studies decision-making for goal-oriented effective communication. We consider an end-to-end status update system where a sensing agent (SA) observes a source, generates and transmits updates to an actuation agent (AA), while the AA takes actions to accomplish a goal at the endpoint. We integrate the push- and pull-based update communication models to obtain a push-and-pull model, which allows the transmission controller at the SA to decide to push an update to the AA and the query controller at the AA to pull updates by raising queries at specific time instances. To gauge effectiveness, we utilize a grade of effectiveness (GoE) metric incorporating updates' freshness, usefulness, and timeliness of actions as qualitative attributes. We then derive effect-aware policies to maximize the expected discounted sum of updates' effectiveness subject to induced costs. The effect-aware policy at the SA considers the potential effectiveness of communicated updates at the endpoint, while at the AA, it accounts for the probabilistic evolution of the source and importance of generated updates. Our results show the proposed push-and-pull model outperforms models solely based on push- or pull-based updates both in terms of efficiency and effectiveness. Additionally, using effect-aware policies at both agents enhances effectiveness compared to periodic and/or probabilistic effect-agnostic policies at either or both agents. △ Less

Submitted 15 January, 2025; v1 submitted 19 July, 2024; originally announced July 2024.

Comments: Submitted for possible publication

Age of Information Versions: a Semantic View of Markov Source Monitoring

Authors: Mehrdad Salimnejad, Marios Kountouris, Anthony Ephremides, Nikolaos Pappas

Abstract: We consider the problem of real-time remote monitoring of a two-state Markov process, where a sensor observes the state of the source and makes a decision on whether to transmit the status updates over an unreliable channel or not. We introduce a modified randomized stationary sampling and transmission policy where the decision to perform sampling occurs probabilistically depending on the current… ▽ More We consider the problem of real-time remote monitoring of a two-state Markov process, where a sensor observes the state of the source and makes a decision on whether to transmit the status updates over an unreliable channel or not. We introduce a modified randomized stationary sampling and transmission policy where the decision to perform sampling occurs probabilistically depending on the current state of the source and whether the system was in a sync state during the previous time slot or not. We then propose two new performance metrics, coined the Version Innovation Age (VIA) and the Age of Incorrect Version (AoIV) and analyze their performance under the modified randomized stationary and other state-of-the-art sampling and transmission policies. Specifically, we derive closed-form expressions for the distribution and the average of VIA, AoIV, and Age of Incorrect Information (AoII) under these policies. Furthermore, we formulate and solve three constrained optimization problems. The first optimization problem aims to minimize the average VIA subject to constraints on the time-averaged sampling cost and time-averaged reconstruction error. In the second and third problems, the objective is to minimize the average AoIV and AoII, respectively, while considering a constraint on the time-averaged sampling cost. Finally, we compare the performance of various sampling and transmission policies and identify the conditions under which each policy outperforms the others in optimizing the proposed metrics. △ Less

Submitted 20 June, 2024; originally announced June 2024.

Comments: arXiv admin note: substantial text overlap with arXiv:2401.17691

DRACO: Decentralized Asynchronous Federated Learning over Row-Stochastic Wireless Networks

Authors: Eunjeong Jeong, Marios Kountouris

Abstract: Recent developments and emerging use cases, such as smart Internet of Things (IoT) and Edge AI, have sparked considerable interest in the training of neural networks over fully decentralized (serverless) networks. One of the major challenges of decentralized learning is to ensure stable convergence without resorting to strong assumptions applied for each agent regarding data distributions or updat… ▽ More Recent developments and emerging use cases, such as smart Internet of Things (IoT) and Edge AI, have sparked considerable interest in the training of neural networks over fully decentralized (serverless) networks. One of the major challenges of decentralized learning is to ensure stable convergence without resorting to strong assumptions applied for each agent regarding data distributions or updating policies. To address these issues, we propose DRACO, a novel method for decentralized asynchronous Stochastic Gradient Descent (SGD) over row-stochastic gossip wireless networks by leveraging continuous communication. Our approach enables edge devices within decentralized networks to perform local training and model exchanging along a continuous timeline, thereby eliminating the necessity for synchronized timing. The algorithm also features a specific technique of decoupling communication and computation schedules, which empowers complete autonomy for all users and manageable instructions for stragglers. Through a comprehensive convergence analysis, we highlight the advantages of asynchronous and autonomous participation in decentralized optimization. Our numerical experiments corroborate the efficacy of the proposed technique. △ Less

Submitted 29 January, 2025; v1 submitted 19 June, 2024; originally announced June 2024.

Comments: This paper has been submitted to a peer-reviewed journal and is currently under review

Integrated Sensing and Communications with Affine Frequency Division Multiplexing

Authors: Ali Bemani, Nassar Ksairi, Marios Kountouris

Abstract: Integrated sensing and communications (ISAC) is regarded as a key technology in next-generation (6G) mobile communication systems. Affine frequency division multiplexing (AFDM) is a recently proposed waveform that achieves optimal diversity gain in high mobility scenarios and has appealing properties in high-frequency communication. In this letter, we present an AFDM-based ISAC system. We first sh… ▽ More Integrated sensing and communications (ISAC) is regarded as a key technology in next-generation (6G) mobile communication systems. Affine frequency division multiplexing (AFDM) is a recently proposed waveform that achieves optimal diversity gain in high mobility scenarios and has appealing properties in high-frequency communication. In this letter, we present an AFDM-based ISAC system. We first show that in order to identify all delay and Doppler components associated with the propagation medium, either the full AFDM signal or only its pilot part consisting of one discrete affine Fourier transform (DAFT) domain symbol and its guard interval can be used. Our results show that using one pilot symbol achieves almost the same sensing performance as using the entire AFDM frame. Furthermore, due to the chirp nature of AFDM, sensing with one pilot provides a unique feature allowing for simple self-interference cancellation, thus avoiding the need for expensive full duplex methods. △ Less

Submitted 26 February, 2024; originally announced February 2024.

Comments: Published in IEEE Wireless Communications Letters, 2024

Version Innovation Age and Age of Incorrect Version for Monitoring Markovian Sources

Authors: Mehrdad Salimnejad, Marios Kountouris, Anthony Ephremides, Nikolaos Pappas

Abstract: In this paper, we propose two new performance metrics, coined the Version Innovation Age (VIA) and the Age of Incorrect Version (AoIV) for real-time monitoring of a two-state Markov process over an unreliable channel. We analyze their performance under the change-aware, semantics-aware, and randomized stationary sampling and transmission policies. We derive closed-form expressions for the distribu… ▽ More In this paper, we propose two new performance metrics, coined the Version Innovation Age (VIA) and the Age of Incorrect Version (AoIV) for real-time monitoring of a two-state Markov process over an unreliable channel. We analyze their performance under the change-aware, semantics-aware, and randomized stationary sampling and transmission policies. We derive closed-form expressions for the distribution and the average of VIA, AoIV, and AoII for these policies. We then formulate and solve an optimization problem to minimize the average VIA, subject to constraints on the time-averaged sampling cost and time-averaged reconstruction error. Finally, we compare the performance of various sampling and transmission policies and identify the conditions under which each policy outperforms the others in optimizing the proposed metrics. △ Less

Submitted 31 January, 2024; originally announced January 2024.

A Latent Space Metric for Enhancing Prediction Confidence in Earth Observation Data

Authors: Ioannis Pitsiorlas, Argyro Tsantalidou, George Arvanitakis, Marios Kountouris, Charalambos Kontoes

Abstract: This study presents a new approach for estimating confidence in machine learning model predictions, specifically in regression tasks utilizing Earth Observation (EO) data, with a particular focus on mosquito abundance (MA) estimation. We take advantage of a Variational AutoEncoder architecture, to derive a confidence metric by the latent space representations of EO datasets. This methodology is pi… ▽ More This study presents a new approach for estimating confidence in machine learning model predictions, specifically in regression tasks utilizing Earth Observation (EO) data, with a particular focus on mosquito abundance (MA) estimation. We take advantage of a Variational AutoEncoder architecture, to derive a confidence metric by the latent space representations of EO datasets. This methodology is pivotal in establishing a correlation between the Euclidean distance in latent representations and the Absolute Error (AE) in individual MA predictions. Our research focuses on EO datasets from the Veneto region in Italy and the Upper Rhine Valley in Germany, targeting areas significantly affected by mosquito populations. A key finding is a notable correlation of 0.46 between the AE of MA predictions and the proposed confidence metric. This correlation signifies a robust, new metric for quantifying the reliability and enhancing the trustworthiness of the AI model's predictions in the context of both EO data analysis and mosquito abundance studies. △ Less

Submitted 11 June, 2024; v1 submitted 30 January, 2024; originally announced January 2024.

Copula-based Estimation of Continuous Sources for a Class of Constrained Rate-Distortion-Functions

Authors: Giuseppe Serra, Photios A. Stavrou, Marios Kountouris

Abstract: We present a new method to estimate the rate-distortion-perception function in the perfect realism regime (PR-RDPF), for multivariate continuous sources subject to a single-letter average distortion constraint. The proposed approach is not only able to solve the specific problem but also two related problems: the entropic optimal transport (EOT) and the output-constrained rate-distortion function… ▽ More We present a new method to estimate the rate-distortion-perception function in the perfect realism regime (PR-RDPF), for multivariate continuous sources subject to a single-letter average distortion constraint. The proposed approach is not only able to solve the specific problem but also two related problems: the entropic optimal transport (EOT) and the output-constrained rate-distortion function (OC-RDF), of which the PR-RDPF represents a special case. Using copula distributions, we show that the OC-RDF can be cast as an I-projection problem on a convex set, based on which we develop a parametric solution of the optimal projection proving that its parameters can be estimated, up to an arbitrary precision, via the solution of a convex program. Subsequently, we propose an iterative scheme via gradient methods to estimate the convex program. Lastly, we characterize a Shannon lower bound (SLB) for the PR-RDPF under a mean squared error (MSE) distortion constraint. We support our theoretical findings with numerical examples by assessing the estimation performance of our iterative scheme using the PR-RDPF with the obtained SLB for various sources. △ Less

Submitted 30 January, 2024; originally announced January 2024.

How to Collaborate: Towards Maximizing the Generalization Performance in Cross-Silo Federated Learning

Authors: Yuchang Sun, Marios Kountouris, Jun Zhang

Abstract: Federated learning (FL) has attracted vivid attention as a privacy-preserving distributed learning framework. In this work, we focus on cross-silo FL, where clients become the model owners after training and are only concerned about the model's generalization performance on their local data. Due to the data heterogeneity issue, asking all the clients to join a single FL training process may result… ▽ More Federated learning (FL) has attracted vivid attention as a privacy-preserving distributed learning framework. In this work, we focus on cross-silo FL, where clients become the model owners after training and are only concerned about the model's generalization performance on their local data. Due to the data heterogeneity issue, asking all the clients to join a single FL training process may result in model performance degradation. To investigate the effectiveness of collaboration, we first derive a generalization bound for each client when collaborating with others or when training independently. We show that the generalization performance of a client can be improved only by collaborating with other clients that have more training data and similar data distribution. Our analysis allows us to formulate a client utility maximization problem by partitioning clients into multiple collaborating groups. A hierarchical clustering-based collaborative training (HCCT) scheme is then proposed, which does not need to fix in advance the number of groups. We further analyze the convergence of HCCT for general non-convex loss functions which unveils the effect of data similarity among clients. Extensive simulations show that HCCT achieves better generalization performance than baseline schemes, whereas it degenerates to independent training and conventional FL in specific scenarios. △ Less

Submitted 28 November, 2024; v1 submitted 24 January, 2024; originally announced January 2024.

Goal-Oriented Multiple Access Connectivity for Networked Intelligent Systems

Authors: Pouya Agheli, Nikolaos Pappas, Marios Kountouris

Abstract: We design a self-decision goal-oriented multiple access scheme, where sensing agents observe a common event and individually decide to communicate the event's attributes as updates to the monitoring agents, to satisfy a certain goal. Decisions are based on the usefulness of updates, generated under uniform, change- and semantics-aware acquisition, as well as statistics and updates of other agents.… ▽ More We design a self-decision goal-oriented multiple access scheme, where sensing agents observe a common event and individually decide to communicate the event's attributes as updates to the monitoring agents, to satisfy a certain goal. Decisions are based on the usefulness of updates, generated under uniform, change- and semantics-aware acquisition, as well as statistics and updates of other agents. We obtain optimal activation probabilities and threshold criteria for decision-making under all schemes, maximizing a grade of effectiveness metric. Alongside studying the effect of different parameters on effectiveness, our simulation results show that the self-decision scheme may attain at least 92% of optimal performance. △ Less

Submitted 14 June, 2024; v1 submitted 19 January, 2024; originally announced January 2024.

Comments: Accepted for publication in IEEE Communications Letters

On the Computation of the Gaussian Rate-Distortion-Perception Function

Authors: Giuseppe Serra, Photios A. Stavrou, Marios Kountouris

Abstract: In this paper, we study the computation of the rate-distortion-perception function (RDPF) for a multivariate Gaussian source under mean squared error (MSE) distortion and, respectively, Kullback-Leibler divergence, geometric Jensen-Shannon divergence, squared Hellinger distance, and squared Wasserstein-2 distance perception metrics. To this end, we first characterize the analytical bounds of the s… ▽ More In this paper, we study the computation of the rate-distortion-perception function (RDPF) for a multivariate Gaussian source under mean squared error (MSE) distortion and, respectively, Kullback-Leibler divergence, geometric Jensen-Shannon divergence, squared Hellinger distance, and squared Wasserstein-2 distance perception metrics. To this end, we first characterize the analytical bounds of the scalar Gaussian RDPF for the aforementioned divergence functions, also providing the RDPF-achieving forward "test-channel" realization. Focusing on the multivariate case, we establish that, for tensorizable distortion and perception metrics, the optimal solution resides on the vector space spanned by the eigenvector of the source covariance matrix. Consequently, the multivariate optimization problem can be expressed as a function of the scalar Gaussian RDPFs of the source marginals, constrained by global distortion and perception levels. Leveraging this characterization, we design an alternating minimization scheme based on the block nonlinear Gauss-Seidel method, which optimally solves the problem while identifying the Gaussian RDPF-achieving realization. Furthermore, the associated algorithmic embodiment is provided, as well as the convergence and the rate of convergence characterization. Lastly, for the "perfect realism" regime, the analytical solution for the multivariate Gaussian RDPF is obtained. We corroborate our results with numerical simulations and draw connections to existing results. △ Less

Submitted 15 November, 2023; originally announced November 2023.

Comments: This paper has been submitted for journal publication

Effective Communication: When to Pull Updates?

Authors: Pouya Agheli, Nikolaos Pappas, Petar Popovski, Marios Kountouris

Abstract: We study a pull-based communication system where a sensing agent updates an actuation agent using a query control policy, which is adjusted in the evolution of an observed information source and the usefulness of each update for achieving a specific goal. For that, a controller decides whether to pull an update at each slot, predicting what is probably occurring at the source and how much effectiv… ▽ More We study a pull-based communication system where a sensing agent updates an actuation agent using a query control policy, which is adjusted in the evolution of an observed information source and the usefulness of each update for achieving a specific goal. For that, a controller decides whether to pull an update at each slot, predicting what is probably occurring at the source and how much effective impact that update could have at the endpoint. Thus, temporal changes in the source evolution could modify the query arrivals so as to capture important updates. The amount of impact is determined by a grade of effectiveness (GoE) metric, which incorporates both freshness and usefulness attributes of the communicated updates. Applying an iterative algorithm, we derive query decisions that maximize the long-term average GoE for the communicated packets, subject to cost constraints. Our analytical and numerical results show that the proposed query policy exhibits higher effectiveness than existing periodic and probabilistic query policies for a wide range of query arrival rates. △ Less

Submitted 14 February, 2024; v1 submitted 10 November, 2023; originally announced November 2023.

Comments: Accepted to be presented in the IEEE ICC 2024

State-aware Real-time Tracking and Remote Reconstruction of a Markov Source

Authors: Mehrdad Salimnejad, Marios Kountouris, Nikolaos Pappas

Abstract: The problem of real-time remote tracking and reconstruction of a two-state Markov process is considered here. A transmitter sends samples from an observed information source to a remote monitor over an unreliable wireless channel. The receiver, in turn, performs an action according to the state of the reconstructed source. We propose a state-aware randomized stationary sampling and transmission po… ▽ More The problem of real-time remote tracking and reconstruction of a two-state Markov process is considered here. A transmitter sends samples from an observed information source to a remote monitor over an unreliable wireless channel. The receiver, in turn, performs an action according to the state of the reconstructed source. We propose a state-aware randomized stationary sampling and transmission policy which accounts for the importance of different states of the information source, and their impact on the goal of the communication process. We then analyze the performance of the proposed policy, and compare it with existing goal-oriented joint sampling and transmission policies, with respect to a set of performance metrics. Specifically, we study the real-time reconstruction error, the cost of actuation error, the consecutive error, and a new metric, coined importance-aware consecutive error. In addition, we formulate and solve a constrained optimization problem that aims to obtain the optimal sampling probabilities that minimize the average cost of actuation error. Our results show that in the scenario of constrained sampling generation, the optimal state-aware randomized stationary policy outperforms all other sampling policies for fast evolving sources, and, under certain conditions, for slowly varying sources. Otherwise, a semantics-aware policy performs better only when the source is slowly varying. △ Less

Submitted 21 September, 2023; originally announced September 2023.

Comments: arXiv admin note: text overlap with arXiv:2302.13927

An Information-Theoretic Analog of the Twin Paradox

Authors: Mladen Kovačević, Iosif Pinelis, Marios Kountouris

Abstract: We revisit the familiar scenario involving two parties in relative motion, in which Alice stays at rest while Bob goes on a journey at speed $βc$ along an arbitrary trajectory and reunites with Alice after a certain period of time. It is a well-known consequence of special relativity that the time that passes until they meet again is different for the two parties and is shorter in Bob's frame by a… ▽ More We revisit the familiar scenario involving two parties in relative motion, in which Alice stays at rest while Bob goes on a journey at speed $βc$ along an arbitrary trajectory and reunites with Alice after a certain period of time. It is a well-known consequence of special relativity that the time that passes until they meet again is different for the two parties and is shorter in Bob's frame by a factor of $\sqrt{1-β^2}$. We investigate how this asymmetry manifests from an information-theoretic viewpoint. Assuming that Alice and Bob transmit signals of equal average power to each other during the whole journey, and that additive white Gaussian noise is present at both sides, we show that the maximum number of bits per second that Alice can transmit reliably to Bob is always higher than the one Bob can transmit to Alice. Equivalently, the energy per bit invested by Alice is lower than that invested by Bob, meaning that the traveler is less efficient from the communication perspective, as conjectured by Jarett and Cover. △ Less

Submitted 16 April, 2024; v1 submitted 21 September, 2023; originally announced September 2023.

Comments: To appear in Europhysics Letters (EPL)

MSC Class: 83A05; 94A24; 94A40

Journal ref: EPL, vol. 146, no. 4, art. no. 42002, 2024

Semantic Filtering and Source Coding in Distributed Wireless Monitoring Systems

Authors: Pouya Agheli, Nikolaos Pappas, Marios Kountouris

Abstract: The problem of goal-oriented semantic filtering and timely source coding in multiuser communication systems is considered here. We study a distributed monitoring system in which multiple information sources, each observing a physical process, provide status update packets to multiple monitors having heterogeneous goals. Two semantic filtering schemes are first proposed as a means to admit or drop… ▽ More The problem of goal-oriented semantic filtering and timely source coding in multiuser communication systems is considered here. We study a distributed monitoring system in which multiple information sources, each observing a physical process, provide status update packets to multiple monitors having heterogeneous goals. Two semantic filtering schemes are first proposed as a means to admit or drop arrival packets based on their goal-dependent importance, which is a function of the intrinsic and extrinsic attributes of information and the probability of occurrence of each realization. Admitted packets at each sensor are then encoded and transmitted over block-fading wireless channels so that served monitors can timely fulfill their goals. A truncated error control scheme is derived, which allows transmitters to drop or retransmit undelivered packets based on their significance. Then, we formulate the timely source encoding optimization problem and analytically derive the optimal codeword lengths assigned to the admitted packets which maximize a weighted sum of semantic utility functions for all pairs of communicating sensors and monitors. Our analytical and numerical results provide the optimal design parameters for different arrival rates and highlight the improvement in timely status update delivery using the proposed semantic filtering, source coding, and error control schemes. △ Less

Submitted 14 February, 2024; v1 submitted 19 May, 2023; originally announced May 2023.

Comments: Accepted to be published in IEEE Transactions on Communications

Indirect Rate Distortion Functions with $f$-Separable Distortion Criterion

Authors: Photios A. Stavrou, Yanina Shkel, Marios Kountouris

Abstract: We consider a remote source coding problem subject to a {distortion function}. Contrary to the use of the classical separable distortion criterion, herein we consider the more general, $f$-separable distortion measure and study its implications on the characterization of the minimum achievable rates (also called $f$-separable indirect rate distortion function (iRDF)) under both excess and average… ▽ More We consider a remote source coding problem subject to a {distortion function}. Contrary to the use of the classical separable distortion criterion, herein we consider the more general, $f$-separable distortion measure and study its implications on the characterization of the minimum achievable rates (also called $f$-separable indirect rate distortion function (iRDF)) under both excess and average distortion constraints. First, we provide a single-letter characterization of the optimal rates subject to an excess distortion using properties of the $f$-separable distortion. Our main result is a single-letter characterization of the $f$-separable iRDF subject to an average distortion constraint. As a consequence of the previous results, we also show a series of equalities that hold using either indirect or classical RDF under $f$-separable excess or average distortions. We corroborate our results with two application examples in which new closed-form solutions are derived, and based on these, we also recover known special cases. △ Less

Submitted 17 May, 2023; originally announced May 2023.

Comments: 6 pages, 2 figures, accepted for publication to the proceedings of ISIT 2023

MSC Class: 94A17; 94A34; 68P30; 94A15 ACM Class: H.1.1; G.3

Computation of Rate-Distortion-Perception Function under f-Divergence Perception Constraints

Authors: Giuseppe Serra, Photios A. Stavrou, Marios Kountouris

Abstract: In this paper, we study the computation of the rate-distortion-perception function (RDPF) for discrete memoryless sources subject to a single-letter average distortion constraint and a perception constraint that belongs to the family of f-divergences. For that, we leverage the fact that RDPF, assuming mild regularity conditions on the perception constraint, forms a convex programming problem. We f… ▽ More In this paper, we study the computation of the rate-distortion-perception function (RDPF) for discrete memoryless sources subject to a single-letter average distortion constraint and a perception constraint that belongs to the family of f-divergences. For that, we leverage the fact that RDPF, assuming mild regularity conditions on the perception constraint, forms a convex programming problem. We first develop parametric characterizations of the optimal solution and utilize them in an alternating minimization approach for which we prove convergence guarantees. The resulting structure of the iterations of the alternating minimization approach renders the implementation of a generalized Blahut-Arimoto (BA) type of algorithm infeasible. To overcome this difficulty, we propose a relaxed formulation of the structure of the iterations in the alternating minimization approach, which allows for the implementation of an approximate iterative scheme. This approximation is shown, via the derivation of necessary and sufficient conditions, to guarantee convergence to a globally optimal solution. We also provide sufficient conditions on the distortion and the perception constraints which guarantee that our algorithm converges exponentially fast. We corroborate our theoretical results with numerical simulations, and we draw connections with existing results. △ Less

Submitted 8 May, 2023; originally announced May 2023.

Comments: Accepted paper to ISIT 2023 without proofs

Goal-oriented Policies for Cost of Actuation Error Minimization in Wireless Autonomous Systems

Authors: Emmanouil Fountoulakis, Nikolaos Pappas, Marios Kountouris

Abstract: We consider the minimization of the cost of actuation error under resource constraints for real-time tracking in wireless autonomous systems. A transmitter monitors the state of a discrete random process and sends updates to the receiver over an unreliable wireless channel. The receiver takes actions according to the estimated state of the source. For each discrepancy between the real state of the… ▽ More We consider the minimization of the cost of actuation error under resource constraints for real-time tracking in wireless autonomous systems. A transmitter monitors the state of a discrete random process and sends updates to the receiver over an unreliable wireless channel. The receiver takes actions according to the estimated state of the source. For each discrepancy between the real state of the source and the estimated one, we consider a different cost of actuation error. This models the case where some states, and consequently the corresponding actions to be taken, are more important than others. We provide two algorithms: one reaching an optimal solution but of high complexity, and one providing a suboptimal solution but with low complexity. The performance of the two algorithms are quite close as shown by the simulations. △ Less

Submitted 8 March, 2023; originally announced March 2023.

When is Importance Weighting Correction Needed for Covariate Shift Adaptation?

Authors: Davit Gogolashvili, Matteo Zecchin, Motonobu Kanagawa, Marios Kountouris, Maurizio Filippone

Abstract: This paper investigates when the importance weighting (IW) correction is needed to address covariate shift, a common situation in supervised learning where the input distributions of training and test data differ. Classic results show that the IW correction is needed when the model is parametric and misspecified. In contrast, recent results indicate that the IW correction may not be necessary when… ▽ More This paper investigates when the importance weighting (IW) correction is needed to address covariate shift, a common situation in supervised learning where the input distributions of training and test data differ. Classic results show that the IW correction is needed when the model is parametric and misspecified. In contrast, recent results indicate that the IW correction may not be necessary when the model is nonparametric and well-specified. We examine the missing case in the literature where the model is nonparametric and misspecified, and show that the IW correction is needed for obtaining the best approximation of the true unknown function for the test distribution. We do this by analyzing IW-corrected kernel ridge regression, covering a variety of settings, including parametric and nonparametric models, well-specified and misspecified settings, and arbitrary weighting functions. △ Less

Submitted 7 March, 2023; originally announced March 2023.

Real-time Reconstruction of Markov Sources and Remote Actuation over Wireless Channels

Authors: Mehrdad Salimnejad, Marios Kountouris, Nikolaos Pappas

Abstract: In this work, we study the real-time tracking and reconstruction of an information source with the purpose of actuation. A device monitors the state of the information source and transmits status updates to a receiver over a wireless erasure channel. We consider two models for the source, namely an $N$-state Markov chain and an $N$-state Birth-Death Markov process. We investigate several joint sam… ▽ More In this work, we study the real-time tracking and reconstruction of an information source with the purpose of actuation. A device monitors the state of the information source and transmits status updates to a receiver over a wireless erasure channel. We consider two models for the source, namely an $N$-state Markov chain and an $N$-state Birth-Death Markov process. We investigate several joint sampling and transmission policies, including a semantics-aware one, and we study their performance with respect to a set of metrics. Specifically, we investigate the real-time reconstruction error and its variance, the cost of actuation error, the consecutive error, and the cost of memory error. These metrics capture different characteristics of the system performance, such as the impact of erroneous actions and the timing of errors. In addition, we propose a randomized stationary sampling and transmission policy and we derive closed-form expressions for the aforementioned metrics. We then formulate two optimization problems. The first optimization problem aims to minimize the time-averaged reconstruction error subject to time-averaged sampling cost constraint. Then, we compare the optimal randomized stationary policy with uniform, change-aware, and semantics-aware sampling policies. Our results show that in the scenario of constrained sampling generation, the optimal randomized stationary policy outperforms all other sampling policies when the source is rapidly evolving. Otherwise, the semantics-aware policy performs the best. The objective of the second optimization problem is to obtain an optimal sampling policy that minimizes the average consecutive error with a constraint on the time-averaged sampling cost. Based on this, we propose a \emph{wait-then-generate} sampling policy which is simple to implement. △ Less

Submitted 27 February, 2023; originally announced February 2023.

Comments: Submitted for possible journal publication. arXiv admin note: substantial text overlap with arXiv:2302.01132

Personalized Decentralized Federated Learning with Knowledge Distillation

Authors: Eunjeong Jeong, Marios Kountouris

Abstract: Personalization in federated learning (FL) functions as a coordinator for clients with high variance in data or behavior. Ensuring the convergence of these clients' models relies on how closely users collaborate with those with similar patterns or preferences. However, it is generally challenging to quantify similarity under limited knowledge about other users' models given to users in a decentral… ▽ More Personalization in federated learning (FL) functions as a coordinator for clients with high variance in data or behavior. Ensuring the convergence of these clients' models relies on how closely users collaborate with those with similar patterns or preferences. However, it is generally challenging to quantify similarity under limited knowledge about other users' models given to users in a decentralized network. To cope with this issue, we propose a personalized and fully decentralized FL algorithm, leveraging knowledge distillation techniques to empower each device so as to discern statistical distances between local models. Each client device can enhance its performance without sharing local data by estimating the similarity between two intermediate outputs from feeding local samples as in knowledge distillation. Our empirical studies demonstrate that the proposed algorithm improves the test accuracy of clients in fewer iterations under highly non-independent and identically distributed (non-i.i.d.) data distributions and is beneficial to agents with small datasets, even without the need for a central server. △ Less

Submitted 23 February, 2023; originally announced February 2023.

Real-time Remote Reconstruction of a Markov Source and Actuation over Wireless

Authors: Mehrdad Salimnejad, Marios Kountouris, Nikolaos Pappas

Abstract: In this work, we study the problem of real-time tracking and reconstruction of an information source with the purpose of actuation. A device monitors an $N$-state Markov process and transmits status updates to a receiver over a wireless erasure channel. We consider a set of joint sampling and transmission policies, including a semantics-aware one, and we study their performance with respect to rel… ▽ More In this work, we study the problem of real-time tracking and reconstruction of an information source with the purpose of actuation. A device monitors an $N$-state Markov process and transmits status updates to a receiver over a wireless erasure channel. We consider a set of joint sampling and transmission policies, including a semantics-aware one, and we study their performance with respect to relevant metrics. Specifically, we investigate the real-time reconstruction error and its variance, the consecutive error, the cost of memory error, and the cost of actuation error. Furthermore, we propose a randomized stationary sampling and transmission policy and derive closed-form expressions for all aforementioned metrics. We then formulate an optimization problem for minimizing the real-time reconstruction error subject to a sampling cost constraint. Our results show that in the scenario of constrained sampling generation, the optimal randomized stationary policy outperforms all other sampling policies when the source is rapidly evolving. Otherwise, the semantics-aware policy performs the best. △ Less

Submitted 31 March, 2023; v1 submitted 2 February, 2023; originally announced February 2023.

Comments: IEEE ICC 2023 Workshop on Semantic Communications

Multi-User Distributed Computing Via Compressed Sensing

Authors: Ali Khalesi, Sajad Daei, Marios Kountouris, Petros Elia

Abstract: The multi-user linearly-separable distributed computing problem is considered here, in which $N$ servers help to compute the real-valued functions requested by $K$ users, where each function can be written as a linear combination of up to $L$ (generally non-linear) subfunctions. Each server computes a fraction $γ$ of the subfunctions, then communicates a function of its computed outputs to some of… ▽ More The multi-user linearly-separable distributed computing problem is considered here, in which $N$ servers help to compute the real-valued functions requested by $K$ users, where each function can be written as a linear combination of up to $L$ (generally non-linear) subfunctions. Each server computes a fraction $γ$ of the subfunctions, then communicates a function of its computed outputs to some of the users, and then each user collects its received data to recover its desired function. Our goal is to bound the ratio between the computation workload done by all servers over the number of datasets. To this end, we here reformulate the real-valued distributed computing problem into a matrix factorization problem and then into a basic sparse recovery problem, where sparsity implies computational savings. Building on this, we first give a simple probabilistic scheme for subfunction assignment, which allows us to upper bound the optimal normalized computation cost as $γ\leq \frac{K}{N}$ that a generally intractable $\ell_0$-minimization would give. To bypass the intractability of such optimal scheme, we show that if these optimal schemes enjoy $γ\leq - r\frac{K}{N}W^{-1}_{-1}(- \frac{2K}{e N r} )$ (where $W_{-1}(\cdot)$ is the Lambert function and $r$ calibrates the communication between servers and users), then they can actually be derived using a tractable Basis Pursuit $\ell_1$-minimization. This newly-revealed connection between distributed computation and compressed sensing opens up the possibility of designing practical distributed computing algorithms by employing tools and methods from compressed sensing. △ Less

Submitted 9 January, 2023; originally announced January 2023.

Comments: Submitted to ITW2023. arXiv admin note: text overlap with arXiv:2206.11119

Robust Bayesian Learning for Reliable Wireless AI: Framework and Applications

Authors: Matteo Zecchin, Sangwoo Park, Osvaldo Simeone, Marios Kountouris, David Gesbert

Abstract: This work takes a critical look at the application of conventional machine learning methods to wireless communication problems through the lens of reliability and robustness. Deep learning techniques adopt a frequentist framework, and are known to provide poorly calibrated decisions that do not reproduce the true uncertainty caused by limitations in the size of the training data. Bayesian learning… ▽ More This work takes a critical look at the application of conventional machine learning methods to wireless communication problems through the lens of reliability and robustness. Deep learning techniques adopt a frequentist framework, and are known to provide poorly calibrated decisions that do not reproduce the true uncertainty caused by limitations in the size of the training data. Bayesian learning, while in principle capable of addressing this shortcoming, is in practice impaired by model misspecification and by the presence of outliers. Both problems are pervasive in wireless communication settings, in which the capacity of machine learning models is subject to resource constraints and training data is affected by noise and interference. In this context, we explore the application of the framework of robust Bayesian learning. After a tutorial-style introduction to robust Bayesian learning, we showcase the merits of robust Bayesian learning on several important wireless communication problems in terms of accuracy, calibration, and robustness to outliers and misspecification. △ Less

Submitted 1 July, 2022; originally announced July 2022.

Comments: Submitted for publication

Communication-Efficient Distributionally Robust Decentralized Learning

Authors: Matteo Zecchin, Marios Kountouris, David Gesbert

Abstract: Decentralized learning algorithms empower interconnected devices to share data and computational resources to collaboratively train a machine learning model without the aid of a central coordinator. In the case of heterogeneous data distributions at the network nodes, collaboration can yield predictors with unsatisfactory performance for a subset of the devices. For this reason, in this work, we c… ▽ More Decentralized learning algorithms empower interconnected devices to share data and computational resources to collaboratively train a machine learning model without the aid of a central coordinator. In the case of heterogeneous data distributions at the network nodes, collaboration can yield predictors with unsatisfactory performance for a subset of the devices. For this reason, in this work, we consider the formulation of a distributionally robust decentralized learning task and we propose a decentralized single loop gradient descent/ascent algorithm (AD-GDA) to directly solve the underlying minimax optimization problem. We render our algorithm communication-efficient by employing a compressed consensus scheme and we provide convergence guarantees for smooth convex and non-convex loss functions. Finally, we corroborate the theoretical findings with empirical results that highlight AD-GDA's ability to provide unbiased predictors and to greatly improve communication efficiency compared to existing distributionally robust algorithms. △ Less

Submitted 13 January, 2023; v1 submitted 31 May, 2022; originally announced May 2022.

Comments: Published in Transactions on Machine Learning Research (TMLR)

Blind Goal-Oriented Massive Access for Future Wireless Networks

Authors: Sajad Daei, Marios Kountouris

Abstract: Emerging communication networks are envisioned to support massive wireless connectivity of heterogeneous devices with sporadic traffic and diverse requirements in terms of latency, reliability, and bandwidth. Providing multiple access to an increasing number of uncoordinated users and sharing the limited resources become essential in this context. In this work, we revisit the random access (RA) pr… ▽ More Emerging communication networks are envisioned to support massive wireless connectivity of heterogeneous devices with sporadic traffic and diverse requirements in terms of latency, reliability, and bandwidth. Providing multiple access to an increasing number of uncoordinated users and sharing the limited resources become essential in this context. In this work, we revisit the random access (RA) problem and exploit the continuous angular group sparsity feature of wireless channels to propose a novel RA strategy that provides low latency, high reliability, and massive access with limited bandwidth resources in an all-in-one package. To this end, we first design a reconstruction-free goal-oriented optimization problem, which only preserves the angular information required to identify the active devices. To solve this, we propose an alternating direction method of multipliers (ADMM) and derive closed-form expressions for each ADMM step. Then, we design a clustering algorithm that assigns the users in specific groups from which we can identify active stationary devices by their angles. For mobile devices, we propose an alternating minimization algorithm to recover their data and their channel gains simultaneously, which allows us to identify active mobile users. Simulation results show significant performance gains in terms of active user detection and false alarm probabilities as compared to state-of-the-art RA schemes, even with limited number of preambles. Moreover, unlike prior work, the performance of the proposed blind goal-oriented massive access does not depend on the number of devices. △ Less

Submitted 14 May, 2022; originally announced May 2022.

Affine Frequency Division Multiplexing for Next Generation Wireless Communications

Authors: Ali Bemani, Nassar Ksairi, Marios Kountouris

Abstract: Affine Frequency Division Multiplexing (AFDM), a new chirp-based multicarrier waveform for high mobility communications, is introduced here. AFDM is based on discrete affine Fourier transform (DAFT), a generalization of discrete Fourier transform, which is characterized by two parameters that can be adapted to better cope with doubly dispersive channels. First, we derive the explicit input-output… ▽ More Affine Frequency Division Multiplexing (AFDM), a new chirp-based multicarrier waveform for high mobility communications, is introduced here. AFDM is based on discrete affine Fourier transform (DAFT), a generalization of discrete Fourier transform, which is characterized by two parameters that can be adapted to better cope with doubly dispersive channels. First, we derive the explicit input-output relation in the DAFT domain showing the effect of AFDM parameters in the input-output relation. Second, we show how the DAFT parameters underlying AFDM have to be set so that the resulting DAFT domain impulse response conveys a full delay-Doppler representation of the channel. Then, we show analytically that AFDM can achieve full diversity in doubly dispersive channels, where full diversity refers to the number of multipath components separable in either the delay or the Doppler domain, due to its full delay-Doppler representation. Furthermore, we present a low complexity detection method taking advantage of zero-padding. We also propose an embedded pilot-aided channel estimation scheme for AFDM, in which both channel estimation and data detection are performed within the same AFDM frame. Finally, simulations corroborate the validity of our analytical results and show the significant performance gains of AFDM over state-of-the-art multicarrier schemes in high mobility scenarios. △ Less

Submitted 3 May, 2022; v1 submitted 27 April, 2022; originally announced April 2022.

Comments: Submitted to IEEE Transactions on Wireless Communications

Semantics-Aware Source Coding in Status Update Systems

Authors: Pouya Agheli, Nikolaos Pappas, Marios Kountouris

Abstract: We consider a communication system in which the destination receives status updates from an information source that observes a physical process. The transmitter performs semantics-empowered filtering as a means to send only the most "important" samples to the receiver in a timely manner. As a first step, we explore a simple policy where the transmitter selects to encode only a fraction of the leas… ▽ More We consider a communication system in which the destination receives status updates from an information source that observes a physical process. The transmitter performs semantics-empowered filtering as a means to send only the most "important" samples to the receiver in a timely manner. As a first step, we explore a simple policy where the transmitter selects to encode only a fraction of the least frequent realizations of the observed random phenomenon, treating the remaining ones as not informative. For this timely source coding problem, we derive the optimal codeword lengths in the sense of maximizing a semantics-aware utility function and minimizing a quadratic average length cost. Our numerical results show the optimal number of updates to transmit for different arrival rates and encoding costs and corroborate that semantic filtering results in higher performance in terms of timely delivery of important updates. △ Less

Submitted 3 April, 2022; v1 submitted 16 March, 2022; originally announced March 2022.

Low complexity equalization for AFDM in doubly dispersive channels

Authors: Ali Bemani, Nassar Ksairi, Marios Kountouris

Abstract: Affine Frequency Division Multiplexing (AFDM), which is based on discrete affine Fourier transform (DAFT), has recently been proposed for reliable communication in high-mobility scenarios. Two low complexity detectors for AFDM are introduced here. Approximating the channel matrix as a band matrix via placing null symbols in the AFDM frame in the DAFT domain, a low complexity MMSE detection is prop… ▽ More Affine Frequency Division Multiplexing (AFDM), which is based on discrete affine Fourier transform (DAFT), has recently been proposed for reliable communication in high-mobility scenarios. Two low complexity detectors for AFDM are introduced here. Approximating the channel matrix as a band matrix via placing null symbols in the AFDM frame in the DAFT domain, a low complexity MMSE detection is proposed by means of the $\rm{LDL}$ factorization. Furthermore, exploiting the sparsity of the channel matrix, we propose a low complexity iterative decision feedback equalizer (DFE) based on weighted maximal ratio combining (MRC), which extracts and combines the received multipath components of the transmitted symbols in the DAFT domain. Simulation results show that the proposed detectors have similar performance, while weighted MRC-based DFE has lower complexity than band-matrix-approximation LMMSE when the channel impulse response has gaps. △ Less

Submitted 7 March, 2022; v1 submitted 3 March, 2022; originally announced March 2022.

Comments: Accepted in 2022 IEEE International Conference on Acoustics, Speech and Signal Processing, IEEE ICASSP 2022

Robust PAC$^m$: Training Ensemble Models Under Misspecification and Outliers

Authors: Matteo Zecchin, Sangwoo Park, Osvaldo Simeone, Marios Kountouris, David Gesbert

Abstract: Standard Bayesian learning is known to have suboptimal generalization capabilities under misspecification and in the presence of outliers. PAC-Bayes theory demonstrates that the free energy criterion minimized by Bayesian learning is a bound on the generalization error for Gibbs predictors (i.e., for single models drawn at random from the posterior) under the assumption of sampling distributions u… ▽ More Standard Bayesian learning is known to have suboptimal generalization capabilities under misspecification and in the presence of outliers. PAC-Bayes theory demonstrates that the free energy criterion minimized by Bayesian learning is a bound on the generalization error for Gibbs predictors (i.e., for single models drawn at random from the posterior) under the assumption of sampling distributions uncontaminated by outliers. This viewpoint provides a justification for the limitations of Bayesian learning when the model is misspecified, requiring ensembling, and when data is affected by outliers. In recent work, PAC-Bayes bounds -- referred to as PAC$^m$ -- were derived to introduce free energy metrics that account for the performance of ensemble predictors, obtaining enhanced performance under misspecification. This work presents a novel robust free energy criterion that combines the generalized logarithm score function with PAC$^m$ ensemble bounds. The proposed free energy training criterion produces predictive distributions that are able to concurrently counteract the detrimental effects of misspecification -- with respect to both likelihood and prior distribution -- and outliers. △ Less

Submitted 23 April, 2023; v1 submitted 3 March, 2022; originally announced March 2022.

UAV-Aided Decentralized Learning over Mesh Networks

Authors: Matteo Zecchin, David Gesbert, Marios Kountouris

Abstract: Decentralized learning empowers wireless network devices to collaboratively train a machine learning (ML) model relying solely on device-to-device (D2D) communication. It is known that the convergence speed of decentralized optimization algorithms severely depends on the degree of the network connectivity, with denser network topologies leading to shorter convergence time. Consequently, the local… ▽ More Decentralized learning empowers wireless network devices to collaboratively train a machine learning (ML) model relying solely on device-to-device (D2D) communication. It is known that the convergence speed of decentralized optimization algorithms severely depends on the degree of the network connectivity, with denser network topologies leading to shorter convergence time. Consequently, the local connectivity of real world mesh networks, due to the limited communication range of its wireless nodes, undermines the efficiency of decentralized learning protocols, rendering them potentially impracticable. In this work we investigate the role of an unmanned aerial vehicle (UAV), used as flying relay, in facilitating decentralized learning procedures in such challenging conditions. We propose an optimized UAV trajectory, that is defined as a sequence of waypoints that the UAV visits sequentially in order to transfer intelligence across sparsely connected group of users. We then provide a series of experiments highlighting the essential role of UAVs in the context of decentralized learning over mesh networks. △ Less

Submitted 31 May, 2022; v1 submitted 2 March, 2022; originally announced March 2022.

Comments: Accepted to the 30th European Signal Processing Conference, EUSIPCO 2022

Towards Disentangling Information Paths with Coded ResNeXt

Authors: Apostolos Avranas, Marios Kountouris

Abstract: The conventional, widely used treatment of deep learning models as black boxes provides limited or no insights into the mechanisms that guide neural network decisions. Significant research effort has been dedicated to building interpretable models to address this issue. Most efforts either focus on the high-level features associated with the last layers, or attempt to interpret the output of a sin… ▽ More The conventional, widely used treatment of deep learning models as black boxes provides limited or no insights into the mechanisms that guide neural network decisions. Significant research effort has been dedicated to building interpretable models to address this issue. Most efforts either focus on the high-level features associated with the last layers, or attempt to interpret the output of a single layer. In this paper, we take a novel approach to enhance the transparency of the function of the whole network. We propose a neural network architecture for classification, in which the information that is relevant to each class flows through specific paths. These paths are designed in advance before training leveraging coding theory and without depending on the semantic similarities between classes. A key property is that each path can be used as an autonomous single-purpose model. This enables us to obtain, without any additional training and for any class, a lightweight binary classifier that has at least $60\%$ fewer parameters than the original network. Furthermore, our coding theory based approach allows the neural network to make early predictions at intermediate layers during inference, without requiring its full evaluation. Remarkably, the proposed architecture provides all the aforementioned properties while improving the overall accuracy. We demonstrate these properties on a slightly modified ResNeXt model tested on CIFAR-10/100 and ImageNet-1k. △ Less

Submitted 20 September, 2023; v1 submitted 10 February, 2022; originally announced February 2022.

Asynchronous Decentralized Learning over Unreliable Wireless Networks

Authors: Eunjeong Jeong, Matteo Zecchin, Marios Kountouris

Abstract: Decentralized learning enables edge users to collaboratively train models by exchanging information via device-to-device communication, yet prior works have been limited to wireless networks with fixed topologies and reliable workers. In this work, we propose an asynchronous decentralized stochastic gradient descent (DSGD) algorithm, which is robust to the inherent computation and communication fa… ▽ More Decentralized learning enables edge users to collaboratively train models by exchanging information via device-to-device communication, yet prior works have been limited to wireless networks with fixed topologies and reliable workers. In this work, we propose an asynchronous decentralized stochastic gradient descent (DSGD) algorithm, which is robust to the inherent computation and communication failures occurring at the wireless network edge. We theoretically analyze its performance and establish a non-asymptotic convergence guarantee. Experimental results corroborate our analysis, demonstrating the benefits of asynchronicity and outdated gradient information reuse in decentralized learning over unreliable wireless networks. △ Less

Submitted 2 February, 2022; originally announced February 2022.

Towards Intelligent Context-Aware 6G Security

Authors: André N. Barreto, Stefan Köpsell, Arsenia Chorti, Bertram Poettering, Jens Jelitto, Julia Hesse, Jonathan Boole, Konrad Rieck, Marios Kountouris, Dave Singelee, Kumar Ashwinee

Abstract: Imagine interconnected objects with embedded artificial intelligence (AI), empowered to sense the environment, see it, hear it, touch it, interact with it, and move. As future networks of intelligent objects come to life, tremendous new challenges arise for security, but also new opportunities, allowing to address current, as well as future, pressing needs. In this paper we put forward a roadmap t… ▽ More Imagine interconnected objects with embedded artificial intelligence (AI), empowered to sense the environment, see it, hear it, touch it, interact with it, and move. As future networks of intelligent objects come to life, tremendous new challenges arise for security, but also new opportunities, allowing to address current, as well as future, pressing needs. In this paper we put forward a roadmap towards the realization of a new security paradigm that we articulate as intelligent context-aware security. The premise of this roadmap is that sensing and advanced AI will enable context awareness, which in turn can drive intelligent security mechanisms, such as adaptation and automation of security controls. This concept not only provides immediate answers to burning open questions, in particular with respect to non-functional requirements, such as energy or latency constraints, heterogeneity of radio frequency (RF) technologies and long life span of deployed devices, but also, more importantly, offers a viable answer to scalability by allowing such constraints to be met even in massive connectivity regimes. Furthermore, the proposed roadmap has to be designed ethically, by explicitly placing privacy concerns at its core. The path towards this vision and some of the challenges along the way are discussed in this contribution. △ Less

Submitted 17 December, 2021; originally announced December 2021.

A Perspective on Time towards Wireless 6G

Authors: Petar Popovski, Federico Chiariotti, Kaibin Huang, Anders E. Kalør, Marios Kountouris, Nikolaos Pappas, Beatriz Soret

Abstract: With the advent of 5G technology, the notion of latency got a prominent role in wireless connectivity, serving as a proxy term for addressing the requirements for real-time communication. As wireless systems evolve towards 6G, the ambition to immerse the digital into the physical reality will increase. Besides making the real-time requirements more stringent, this immersion will bring the notions… ▽ More With the advent of 5G technology, the notion of latency got a prominent role in wireless connectivity, serving as a proxy term for addressing the requirements for real-time communication. As wireless systems evolve towards 6G, the ambition to immerse the digital into the physical reality will increase. Besides making the real-time requirements more stringent, this immersion will bring the notions of time, simultaneity, presence, and causality to a new level of complexity. A growing body of research points out that latency is insufficient to parameterize all real-time requirements. Notably, one such requirement that received a significant attention is information freshness, defined through the Age of Information (AoI) and its derivatives. The objective of this article is to investigate the general notion of timing in wireless communication systems and networks and its relation to effective information generation, processing, transmission, and reconstruction at the senders and receivers. We establish a general statistical framework of timing requirements in wireless communication systems, which subsumes both latency and AoI. The framework is made by associating a timing component with the two basic statistical operations, decision and estimation. We first use the framework to present a representative sample of the existing works that deal with timing in wireless communication. Next, it is shown how the framework can be used with different communication models of increasing complexity, starting from the basic Shannon one-way communication model and arriving to communication models for consensus, distributed learning, and inference. Overall, this paper fills an important gap in the literature by providing a systematic treatment of various timing measures in wireless communication and sets the basis for design and optimization for the next-generation real-time systems. △ Less

Submitted 12 May, 2022; v1 submitted 8 June, 2021; originally announced June 2021.

LIDAR and Position-Aided mmWave Beam Selection with Non-local CNNs and Curriculum Training

Authors: Matteo Zecchin, Mahdi Boloursaz Mashhadi, Mikolaj Jankowski, Deniz Gunduz, Marios Kountouris, David Gesbert

Abstract: Efficient millimeter wave (mmWave) beam selection in vehicle-to-infrastructure (V2I) communication is a crucial yet challenging task due to the narrow mmWave beamwidth and high user mobility. To reduce the search overhead of iterative beam discovery procedures, contextual information from light detection and ranging (LIDAR) sensors mounted on vehicles has been leveraged by data-driven methods to p… ▽ More Efficient millimeter wave (mmWave) beam selection in vehicle-to-infrastructure (V2I) communication is a crucial yet challenging task due to the narrow mmWave beamwidth and high user mobility. To reduce the search overhead of iterative beam discovery procedures, contextual information from light detection and ranging (LIDAR) sensors mounted on vehicles has been leveraged by data-driven methods to produce useful side information. In this paper, we propose a lightweight neural network (NN) architecture along with the corresponding LIDAR preprocessing, which significantly outperforms previous works. Our solution comprises multiple novelties that improve both the convergence speed and the final accuracy of the model. In particular, we define a novel loss function inspired by the knowledge distillation idea, introduce a curriculum training approach exploiting line-of-sight (LOS)/non-line-of-sight (NLOS) information, and we propose a non-local attention module to improve the performance for the more challenging NLOS cases. Simulation results on benchmark datasets show that, utilizing solely LIDAR data and the receiver position, our NN-based beam selection scheme can achieve 79.9% throughput of an exhaustive beam sweeping approach without any beam search overhead and 95% by searching among as few as 6 beams. In a typical mmWave V2I scenario, our proposed method considerably reduces the beam search time required to achieve a desired throughput, in comparison with the inverse fingerprinting and hierarchical beam selection schemes. △ Less

Submitted 17 November, 2021; v1 submitted 29 April, 2021; originally announced April 2021.

Comments: Submitted for publication

AFDM: A Full Diversity Next Generation Waveform for High Mobility Communications

Authors: Ali Bemani, Nassar Ksairi, Marios Kountouris

Abstract: We present Affine Frequency Division Multiplexing (AFDM), a new chirp-based multicarrier transceiver scheme for high mobility communications in next-generation wireless systems. AFDM is based on discrete affine Fourier transform (DAFT), a generalization of discrete Fourier transform characterized with two parameters that can be adapted to better cope with doubly dispersive channels. Based on the d… ▽ More We present Affine Frequency Division Multiplexing (AFDM), a new chirp-based multicarrier transceiver scheme for high mobility communications in next-generation wireless systems. AFDM is based on discrete affine Fourier transform (DAFT), a generalization of discrete Fourier transform characterized with two parameters that can be adapted to better cope with doubly dispersive channels. Based on the derived input-output relation, the DAFT parameters underlying AFDM are set in such a way to avoid that time domain channel paths with distinct delays or Doppler frequency shifts overlap in the DAFT domain. The resulting DAFT domain impulse response thus conveys a full delay-Doppler representation of the channel. We show that AFDM can achieve the full diversity of linear time-varying (LTV) channels. Our analytical results are validated through numerical simulations, which evince that AFDM outperforms state-of-the-art multicarrier schemes in terms of bit error rate (BER) in doubly dispersive channels. △ Less

Submitted 22 April, 2021; originally announced April 2021.

Comments: 6 pages, 5 figures, conference

Deep Reinforcement Learning for Resource Constrained Multiclass Scheduling in Wireless Networks

Authors: Apostolos Avranas, Marios Kountouris, Philippe Ciblat

Abstract: The problem of resource constrained scheduling in a dynamic and heterogeneous wireless setting is considered here. In our setup, the available limited bandwidth resources are allocated in order to serve randomly arriving service demands, which in turn belong to different classes in terms of payload data requirement, delay tolerance, and importance/priority. In addition to heterogeneous traffic, an… ▽ More The problem of resource constrained scheduling in a dynamic and heterogeneous wireless setting is considered here. In our setup, the available limited bandwidth resources are allocated in order to serve randomly arriving service demands, which in turn belong to different classes in terms of payload data requirement, delay tolerance, and importance/priority. In addition to heterogeneous traffic, another major challenge stems from random service rates due to time-varying wireless communication channels. Various approaches for scheduling and resource allocation can be used, ranging from simple greedy heuristics and constrained optimization to combinatorics. Those methods are tailored to specific network or application configuration and are usually suboptimal. To this purpose, we resort to deep reinforcement learning (DRL) and propose a distributional Deep Deterministic Policy Gradient (DDPG) algorithm combined with Deep Sets to tackle the aforementioned problem. Furthermore, we present a novel way to use a Dueling Network, which leads to further performance improvement. Our proposed algorithm is tested on both synthetic and real data, showing consistent gains against state-of-the-art conventional methods from combinatorics, optimization, and scheduling metrics. △ Less

Submitted 31 March, 2022; v1 submitted 27 November, 2020; originally announced November 2020.