-
Distributed Optimization with Efficient Communication, Event-Triggered Solution Enhancement, and Operation Stopping
Authors:
Apostolos I. Rikos,
Wei Jiang,
Themistoklis Charalambous,
Karl H. Johansson
Abstract:
In modern large-scale systems with sensor networks and IoT devices it is essential to collaboratively solve complex problems while utilizing network resources efficiently. In our paper we present three distributed optimization algorithms that exhibit efficient communication among nodes. Our first algorithm presents a simple quantized averaged gradient procedure for distributed optimization, which…
▽ More
In modern large-scale systems with sensor networks and IoT devices it is essential to collaboratively solve complex problems while utilizing network resources efficiently. In our paper we present three distributed optimization algorithms that exhibit efficient communication among nodes. Our first algorithm presents a simple quantized averaged gradient procedure for distributed optimization, which is shown to converge to a neighborhood of the optimal solution. Our second algorithm incorporates a novel event-triggered refinement mechanism, which refines the utilized quantization level to enhance the precision of the estimated optimal solution. It enables nodes to terminate their operation according to predefined performance guarantees. Our third algorithm is tailored to operate in environments where each message consists of only a few bits. It incorporates a novel event-triggered mechanism for adjusting the quantizer basis and quantization level, allowing nodes to collaboratively decide operation termination based on predefined performance criteria. We analyze the three algorithms and establish their linear convergence. Finally, an application on distributed sensor fusion for target localization is used to demonstrate their favorable performance compared to existing algorithms in the literature.
△ Less
Submitted 23 April, 2025;
originally announced April 2025.
-
Distributed Optimization with Finite Bit Adaptive Quantization for Efficient Communication and Precision Enhancement
Authors:
Apostolos I. Rikos,
Wei Jiang,
Themistoklis Charalambous,
Karl H. Johansson
Abstract:
In realistic distributed optimization scenarios, individual nodes possess only partial information and communicate over bandwidth constrained channels. For this reason, the development of efficient distributed algorithms is essential. In our paper we addresses the challenge of unconstrained distributed optimization. In our scenario each node's local function exhibits strong convexity with Lipschit…
▽ More
In realistic distributed optimization scenarios, individual nodes possess only partial information and communicate over bandwidth constrained channels. For this reason, the development of efficient distributed algorithms is essential. In our paper we addresses the challenge of unconstrained distributed optimization. In our scenario each node's local function exhibits strong convexity with Lipschitz continuous gradients. The exchange of information between nodes occurs through $3$-bit bandwidth-limited channels (i.e., nodes exchange messages represented by a only $3$-bits). Our proposed algorithm respects the network's bandwidth constraints by leveraging zoom-in and zoom-out operations to adjust quantizer parameters dynamically. We show that during our algorithm's operation nodes are able to converge to the exact optimal solution. Furthermore, we show that our algorithm achieves a linear convergence rate to the optimal solution. We conclude the paper with simulations that highlight our algorithm's unique characteristics.
△ Less
Submitted 18 October, 2024; v1 submitted 9 September, 2024;
originally announced September 2024.
-
Asynchronous Distributed Learning with Quantized Finite-Time Coordination
Authors:
Nicola Bastianello,
Apostolos I. Rikos,
Karl H. Johansson
Abstract:
In this paper we address distributed learning problems over peer-to-peer networks. In particular, we focus on the challenges of quantized communications, asynchrony, and stochastic gradients that arise in this set-up. We first discuss how to turn the presence of quantized communications into an advantage, by resorting to a finite-time, quantized coordination scheme. This scheme is combined with a…
▽ More
In this paper we address distributed learning problems over peer-to-peer networks. In particular, we focus on the challenges of quantized communications, asynchrony, and stochastic gradients that arise in this set-up. We first discuss how to turn the presence of quantized communications into an advantage, by resorting to a finite-time, quantized coordination scheme. This scheme is combined with a distributed gradient descent method to derive the proposed algorithm. Secondly, we show how this algorithm can be adapted to allow asynchronous operations of the agents, as well as the use of stochastic gradients. Finally, we propose a variant of the algorithm which employs zooming-in quantization. We analyze the convergence of the proposed methods and compare them to state-of-the-art alternatives.
△ Less
Submitted 30 August, 2024;
originally announced August 2024.
-
Survey of Distributed Algorithms for Resource Allocation over Multi-Agent Systems
Authors:
Mohammadreza Doostmohammadian,
Alireza Aghasi,
Mohammad Pirani,
Ehsan Nekouei,
Houman Zarrabi,
Reza Keypour,
Apostolos I. Rikos,
Karl H. Johansson
Abstract:
Resource allocation and scheduling in multi-agent systems present challenges due to complex interactions and decentralization. This survey paper provides a comprehensive analysis of distributed algorithms for addressing the distributed resource allocation (DRA) problem over multi-agent systems. It covers a significant area of research at the intersection of optimization, multi-agent systems, and d…
▽ More
Resource allocation and scheduling in multi-agent systems present challenges due to complex interactions and decentralization. This survey paper provides a comprehensive analysis of distributed algorithms for addressing the distributed resource allocation (DRA) problem over multi-agent systems. It covers a significant area of research at the intersection of optimization, multi-agent systems, and distributed consensus-based computing. The paper begins by presenting a mathematical formulation of the DRA problem, establishing a solid foundation for further exploration. Real-world applications of DRA in various domains are examined to underscore the importance of efficient resource allocation, and relevant distributed optimization formulations are presented. The survey then delves into existing solutions for DRA, encompassing linear, nonlinear, primal-based, and dual-formulation-based approaches. Furthermore, this paper evaluates the features and properties of DRA algorithms, addressing key aspects such as feasibility, convergence rate, and network reliability. The analysis of mathematical foundations, diverse applications, existing solutions, and algorithmic properties contributes to a broader comprehension of the challenges and potential solutions for this domain.
△ Less
Submitted 28 January, 2024;
originally announced January 2024.
-
Distributed Optimization via Gradient Descent with Event-Triggered Zooming over Quantized Communication
Authors:
Apostolos I. Rikos,
Wei Jiang,
Themistoklis Charalambous,
Karl H. Johansson
Abstract:
In this paper, we study unconstrained distributed optimization strongly convex problems, in which the exchange of information in the network is captured by a directed graph topology over digital channels that have limited capacity (and hence information should be quantized). Distributed methods in which nodes use quantized communication yield a solution at the proximity of the optimal solution, he…
▽ More
In this paper, we study unconstrained distributed optimization strongly convex problems, in which the exchange of information in the network is captured by a directed graph topology over digital channels that have limited capacity (and hence information should be quantized). Distributed methods in which nodes use quantized communication yield a solution at the proximity of the optimal solution, hence reaching an error floor that depends on the quantization level used; the finer the quantization the lower the error floor. However, it is not possible to determine in advance the optimal quantization level that ensures specific performance guarantees (such as achieving an error floor below a predefined threshold). Choosing a very small quantization level that would guarantee the desired performance, requires {information} packets of very large size, which is not desirable (could increase the probability of packet losses, increase delays, etc) and often not feasible due to the limited capacity of the channels available. In order to obtain a communication-efficient distributed solution and a sufficiently close proximity to the optimal solution, we propose a quantized distributed optimization algorithm that converges in a finite number of steps and is able to adjust the quantization level accordingly. The proposed solution uses a finite-time distributed optimization protocol to find a solution to the problem for a given quantization level in a finite number of steps and keeps refining the quantization level until the difference in the solution between two successive solutions with different quantization levels is below a certain pre-specified threshold.
△ Less
Submitted 8 September, 2023;
originally announced September 2023.
-
Asynchronous Distributed Optimization via ADMM with Efficient Communication
Authors:
Apostolos I. Rikos,
Wei Jiang,
Themistoklis Charalambous,
Karl H. Johansson
Abstract:
In this paper, we focus on an asynchronous distributed optimization problem. In our problem, each node is endowed with a convex local cost function, and is able to communicate with its neighbors over a directed communication network. Furthermore, we assume that the communication channels between nodes have limited bandwidth, and each node suffers from processing delays. We present a distributed al…
▽ More
In this paper, we focus on an asynchronous distributed optimization problem. In our problem, each node is endowed with a convex local cost function, and is able to communicate with its neighbors over a directed communication network. Furthermore, we assume that the communication channels between nodes have limited bandwidth, and each node suffers from processing delays. We present a distributed algorithm which combines the Alternating Direction Method of Multipliers (ADMM) strategy with a finite time quantized averaging algorithm. In our proposed algorithm, nodes exchange quantized valued messages and operate in an asynchronous fashion. More specifically, during every iteration of our algorithm each node (i) solves a local convex optimization problem (for the one of its primal variables), and (ii) utilizes a finite-time quantized averaging algorithm to obtain the value of the second primal variable (since the cost function for the second primal variable is not decomposable). We show that our algorithm converges to the optimal solution at a rate of $O(1/k)$ (where $k$ is the number of time steps) for the case where the local cost function of every node is convex and not-necessarily differentiable. Finally, we demonstrate the operational advantages of our algorithm against other algorithms from the literature.
△ Less
Submitted 8 September, 2023;
originally announced September 2023.
-
Online Distributed Learning with Quantized Finite-Time Coordination
Authors:
Nicola Bastianello,
Apostolos I. Rikos,
Karl H. Johansson
Abstract:
In this paper we consider online distributed learning problems. Online distributed learning refers to the process of training learning models on distributed data sources. In our setting a set of agents need to cooperatively train a learning model from streaming data. Differently from federated learning, the proposed approach does not rely on a central server but only on peer-to-peer communications…
▽ More
In this paper we consider online distributed learning problems. Online distributed learning refers to the process of training learning models on distributed data sources. In our setting a set of agents need to cooperatively train a learning model from streaming data. Differently from federated learning, the proposed approach does not rely on a central server but only on peer-to-peer communications among the agents. This approach is often used in scenarios where data cannot be moved to a centralized location due to privacy, security, or cost reasons. In order to overcome the absence of a central server, we propose a distributed algorithm that relies on a quantized, finite-time coordination protocol to aggregate the locally trained models. Furthermore, our algorithm allows for the use of stochastic gradients during local training. Stochastic gradients are computed using a randomly sampled subset of the local training data, which makes the proposed algorithm more efficient and scalable than traditional gradient descent. In our paper, we analyze the performance of the proposed algorithm in terms of the mean distance from the online solution. Finally, we present numerical results for a logistic regression task.
△ Less
Submitted 31 August, 2023; v1 submitted 13 July, 2023;
originally announced July 2023.
-
Distributed Optimization with Quantized Gradient Descent
Authors:
Apostolos I. Rikos,
Wei Jiang,
Themistoklis Charalambous,
Karl H. Johansson
Abstract:
In this paper, we consider the unconstrained distributed optimization problem, in which the exchange of information in the network is captured by a directed graph topology, thus, nodes can only communicate with their neighbors. Additionally, in our problem, the communication channels among the nodes have limited bandwidth. In order to alleviate this limitation, quantized messages should be exchang…
▽ More
In this paper, we consider the unconstrained distributed optimization problem, in which the exchange of information in the network is captured by a directed graph topology, thus, nodes can only communicate with their neighbors. Additionally, in our problem, the communication channels among the nodes have limited bandwidth. In order to alleviate this limitation, quantized messages should be exchanged among the nodes. For solving this distributed optimization problem, we combine a gradient descent method with a distributed quantized consensus algorithm (which requires the nodes to exchange quantized messages and converges in a finite number of steps). Specifically, at every optimization step, each node (i) performs a gradient descent step (i.e., subtracts the scaled gradient from its current estimate), and (ii) performs a finite-time calculation of the quantized average of every node's estimate in the network. As a consequence, this algorithm approximately mimics the centralized gradient descent algorithm. We show that our algorithm asymptotically converges to a neighborhood of the optimal solution with linear convergence rate. The performance of the proposed algorithm is demonstrated via simple illustrative examples.
△ Less
Submitted 5 December, 2023; v1 submitted 19 November, 2022;
originally announced November 2022.
-
Distributed CPU Scheduling Subject to Nonlinear Constraints
Authors:
Mohammadreza Doostmohammadian,
Alireza Aghasi,
Apostolos I. Rikos,
Andreas Grammenos,
Evangelia Kalyvianaki,
Christoforos N. Hadjicostis,
Karl H. Johansson,
Themistoklis Charalambous
Abstract:
This paper considers a network of collaborating agents for local resource allocation subject to nonlinear model constraints. In many applications, it is required (or desirable) that the solution be anytime feasible in terms of satisfying the sum-preserving global constraint. Motivated by this, sufficient conditions on the nonlinear mapping for anytime feasibility (or non-asymptotic feasibility) ar…
▽ More
This paper considers a network of collaborating agents for local resource allocation subject to nonlinear model constraints. In many applications, it is required (or desirable) that the solution be anytime feasible in terms of satisfying the sum-preserving global constraint. Motivated by this, sufficient conditions on the nonlinear mapping for anytime feasibility (or non-asymptotic feasibility) are addressed in this paper. For the two proposed distributed solutions, one converges over directed weight-balanced networks and the other one over undirected networks. In particular, we elaborate on uniform quantization and discuss the notion of ε-accurate solution, which gives an estimate of how close we can get to the exact optimizer subject to different quantization levels. This work, further, handles general (possibly non-quadratic) strictly convex objective functions with application to CPU allocation among a cloud of data centers via distributed solutions. The results can be used as a coordination mechanism to optimally balance the tasks and CPU resources among a group of networked servers while addressing quantization or limited server capacity.
Index Terms: multi-agent systems, sum-preserving resource allocation, distributed optimization, anytime feasibility
△ Less
Submitted 30 August, 2022;
originally announced August 2022.
-
Finite Time Privacy Preserving Quantized Average Consensus with Transmission Stopping
Authors:
Apostolos I. Rikos,
Christoforos N. Hadjicostis,
Karl H. Johansson
Abstract:
Due to their flexibility, battery powered or energy-harvesting wireless networks are employed in diverse applications. Securing data transmissions between wireless devises is of critical importance in order to avoid privacy-sensitive user data leakage. In this paper, we focus on the scenario where some nodes are curious (but not malicious) and try to identify the initial states of one (or multiple…
▽ More
Due to their flexibility, battery powered or energy-harvesting wireless networks are employed in diverse applications. Securing data transmissions between wireless devises is of critical importance in order to avoid privacy-sensitive user data leakage. In this paper, we focus on the scenario where some nodes are curious (but not malicious) and try to identify the initial states of one (or multiple) other nodes, while some nodes aim to preserve the privacy of their initial states from the curious nodes. We present a privacy preserving finite transmission event-triggered quantized average consensus algorithm. Its operation is suitable for battery-powered or energy-harvesting wireless network since it guarantees (i) efficient (quantized) communication, and (ii) transmission ceasing (which allows preservation of available energy). Furthermore, we present topological conditions under which the proposed algorithm allows nodes to preserve their privacy. We conclude with a comparison of our algorithm against other algorithms in the existing literature.
△ Less
Submitted 17 July, 2022;
originally announced July 2022.
-
Optimal Database Allocation in Finite Time with Efficient Communication and Transmission Stopping over Dynamic Networks
Authors:
Apostolos I. Rikos,
Christoforos N. Hadjicostis,
Karl H. Johansson
Abstract:
In this paper, we focus on the problem of data sharing over a wireless computer network (i.e., a wireless grid). Given a set of available data, we present a distributed algorithm which operates over a dynamically changing network, and allows each node to calculate the optimal allocation of data in a finite number of time steps. We show that our proposed algorithm (i) converges to the optimal solut…
▽ More
In this paper, we focus on the problem of data sharing over a wireless computer network (i.e., a wireless grid). Given a set of available data, we present a distributed algorithm which operates over a dynamically changing network, and allows each node to calculate the optimal allocation of data in a finite number of time steps. We show that our proposed algorithm (i) converges to the optimal solution in finite time with very high probability, and (ii) once the optimal solution is reached, each node is able to cease transmissions without needing knowledge of a global parameter such as the network diameter. Furthermore, our algorithm (i) operates exclusively with quantized values (i.e., each node processes and transmits quantized information), (ii) relies on event-driven updates, and (iii) calculates the optimal solution in the form of a quantized fraction which avoids errors due to quantization. Finally, we demonstrate the operation, performance, and potential advantages of our algorithm over random dynamic networks.
△ Less
Submitted 17 July, 2022;
originally announced July 2022.
-
Distributed Finite Time k-means Clustering with Quantized Communucation and Transmission Stopping
Authors:
Apostolos I. Rikos,
Gabriele Oliva,
Christoforos N. Hadjicostis,
Karl H. Johansson
Abstract:
In this paper, we present a distributed algorithm which implements the $k$-means algorithm in a distributed fashion for multi-agent systems with directed communication links. The goal of $k$-means is to partition the network's agents in mutually exclusive sets (groups) such that agents in the same set have (and possibly share) similar information and are able to calculate a representative value fo…
▽ More
In this paper, we present a distributed algorithm which implements the $k$-means algorithm in a distributed fashion for multi-agent systems with directed communication links. The goal of $k$-means is to partition the network's agents in mutually exclusive sets (groups) such that agents in the same set have (and possibly share) similar information and are able to calculate a representative value for their group.During the operation of our distributed algorithm, each node (i) transmits quantized values in an event-driven fashion, and (ii) exhibits distributed stopping capabilities. Transmitting quantized values leads to more efficient usage of the available bandwidth and reduces the communication bottleneck. Also, in order to preserve available resources, nodes are able to distributively determine whether they can terminate the operation of the proposed algorithm. We characterize the properties of the proposed distributed algorithm and show that its execution (on any static and strongly connected digraph) will partition all agents to mutually exclusive clusters in finite time. We conclude with examples that illustrate the operation, performance, and potential advantages of the proposed algorithm.
△ Less
Submitted 17 July, 2022;
originally announced July 2022.
-
Distributed Optimal Allocation with Quantized Communication and Privacy-Preserving Guarantees
Authors:
Jakob Nylöf,
Apostolos I. Rikos,
Sebin Gracy,
Karl H. Johansson
Abstract:
In this paper, we analyze the problem of optimally allocating resources in a distributed and privacy-preserving manner. We propose a novel distributed optimal resource allocation algorithm with privacy-preserving guarantees, which operates over a directed communication network. Our algorithm converges in finite time and allows each node to process and transmit quantized messages. Our algorithm uti…
▽ More
In this paper, we analyze the problem of optimally allocating resources in a distributed and privacy-preserving manner. We propose a novel distributed optimal resource allocation algorithm with privacy-preserving guarantees, which operates over a directed communication network. Our algorithm converges in finite time and allows each node to process and transmit quantized messages. Our algorithm utilizes a distributed quantized average consensus strategy combined with a privacy-preserving mechanism. We show that the algorithm converges in finite-time, and we prove that, under specific conditions on the network topology, nodes are able to preserve the privacy of their initial state. Finally, to illustrate the results, we consider an example where test kits need to be optimally allocated proportionally to the number of infections in a region. It is shown that the proposed privacy-preserving resource allocation algorithm performs well with an appropriate convergence rate under privacy guarantees.
△ Less
Submitted 17 July, 2022; v1 submitted 29 September, 2021;
originally announced September 2021.
-
Fast Quantized Average Consensus over Static and Dynamic Directed Graphs
Authors:
Apostolos I. Rikos,
Christoforos N. Hadjicostis,
Karl H. Johansson
Abstract:
In this paper we study the distributed average consensus problem in multi-agent systems with directed communication links that are subject to quantized information flow. Specifically, we present and analyze a distributed averaging algorithm which operates exclusively with quantized values (i.e., the information stored, processed and exchanged between neighboring agents is subject to deterministic…
▽ More
In this paper we study the distributed average consensus problem in multi-agent systems with directed communication links that are subject to quantized information flow. Specifically, we present and analyze a distributed averaging algorithm which operates exclusively with quantized values (i.e., the information stored, processed and exchanged between neighboring agents is subject to deterministic uniform quantization) and relies on event-driven updates (e.g., to reduce energy consumption, communication bandwidth, network congestion, and/or processor usage). The main idea of the proposed algorithm is that each node (i) models its initial state as two quantized fractions which have numerators equal to the node's initial state and denominators equal to one, and (ii) transmits one fraction randomly while it keeps the other stored. Then, every time it receives one or more fractions, it averages their numerators with the numerator of the fraction it stored, and then transmits them to randomly selected out-neighbors. We characterize the properties of the proposed distributed algorithm and show that its execution, on any static and strongly connected digraph, allows each agent to reach in finite time a fixed state that is equal (within one quantisation level) to the average of the initial states. We extend the operation of the algorithm to achieve finite-time convergence in the presence of a dynamic directed communication topology subject to some connectivity conditions. Finally, we provide examples to illustrate the operation, performance, and potential advantages of the proposed algorithm. We compare against state-of-the-art quantized average consensus algorithms and show that our algorithm's convergence speed significantly outperforms most existing protocols.
△ Less
Submitted 8 March, 2021;
originally announced March 2021.
-
Distributed Event-Triggered Algorithms for Finite-Time Privacy-Preserving Quantized Average Consensus
Authors:
Apostolos I. Rikos,
Themistoklis Charalambous,
Karl H. Johansson,
Christoforos N. Hadjicostis
Abstract:
In this paper, we consider the problem of privacy preservation in the average consensus problem when communication among nodes is quantized. More specifically, we consider a setting where some nodes in the network are curious but not malicious and they try to identify the initial states of other nodes based on the data they receive during their operation (without interfering in the computation in…
▽ More
In this paper, we consider the problem of privacy preservation in the average consensus problem when communication among nodes is quantized. More specifically, we consider a setting where some nodes in the network are curious but not malicious and they try to identify the initial states of other nodes based on the data they receive during their operation (without interfering in the computation in any other way), while some nodes in the network want to ensure that their initial states cannot be inferred exactly by the curious nodes. We propose two privacy-preserving event-triggered quantized average consensus algorithms that can be followed by any node wishing to maintain its privacy and not reveal the initial state it contributes to the average computation. Every node in the network (including the curious nodes) is allowed to execute a privacy-preserving algorithm or its underlying average consensus algorithm. Under certain topological conditions, both algorithms allow the nodes who adopt privacypreserving protocols to preserve the privacy of their initial quantized states and at the same time to obtain, after a finite number of steps, the exact average of the initial states.
△ Less
Submitted 12 February, 2021;
originally announced February 2021.
