-
Momentum-based Accelerated Algorithm for Distributed Optimization under Sector-Bound Nonlinearity
Authors:
Mohammadreza Doostmohammadian,
Hamid R. Rabiee
Abstract:
Distributed optimization advances centralized machine learning methods by enabling parallel and decentralized learning processes over a network of computing nodes. This work provides an accelerated consensus-based distributed algorithm for locally non-convex optimization using the gradient-tracking technique. The proposed algorithm (i) improves the convergence rate by adding momentum towards the o…
▽ More
Distributed optimization advances centralized machine learning methods by enabling parallel and decentralized learning processes over a network of computing nodes. This work provides an accelerated consensus-based distributed algorithm for locally non-convex optimization using the gradient-tracking technique. The proposed algorithm (i) improves the convergence rate by adding momentum towards the optimal state using the heavy-ball method, while (ii) addressing general sector-bound nonlinearities over the information-sharing network. The link nonlinearity includes any sign-preserving odd sector-bound mapping, for example, log-scale data quantization or clipping in practical applications. For admissible momentum and gradient-tracking parameters, using perturbation theory and eigen-spectrum analysis, we prove convergence even in the presence of sector-bound nonlinearity and for locally non-convex cost functions. Further, in contrast to most existing weight-stochastic algorithms, we adopt weight-balanced (WB) network design. This WB design and perturbation-based analysis allow to handle dynamic directed network of agents to address possible time-varying setups due to link failures or packet drops.
△ Less
Submitted 28 June, 2025;
originally announced June 2025.
-
On the Design of Resilient Distributed Single Time-Scale Estimators: A Graph-Theoretic Approach
Authors:
Mohammadreza Doostmohammadian,
Mohammad Pirani
Abstract:
Distributed estimation in interconnected systems has gained increasing attention due to its relevance in diverse applications such as sensor networks, autonomous vehicles, and cloud computing. In real practice, the sensor network may suffer from communication and/or sensor failures. This might be due to cyber-attacks, faults, or environmental conditions. Distributed estimation resilient to such co…
▽ More
Distributed estimation in interconnected systems has gained increasing attention due to its relevance in diverse applications such as sensor networks, autonomous vehicles, and cloud computing. In real practice, the sensor network may suffer from communication and/or sensor failures. This might be due to cyber-attacks, faults, or environmental conditions. Distributed estimation resilient to such conditions is the topic of this paper. By representing the sensor network as a graph and exploiting its inherent structural properties, we introduce novel techniques that enhance the robustness of distributed estimators. As compared to the literature, the proposed estimator (i) relaxes the network connectivity of most existing single time-scale estimators and (ii) reduces the communication load of the existing double time-scale estimators by avoiding the inner consensus loop.
On the other hand, the sensors might be subject to faults or attacks, resulting in biased measurements. Removing these sensor data may result in observability loss. Therefore, we propose resilient design on the definitions of $q$-node-connectivity and $q$-link-connectivity, which capture robust strong-connectivity under link or sensor node failure. By proper design of the sensor network, we prove Schur stability of the proposed distributed estimation protocol under failure of up to $q$ sensors or $q$ communication links.
△ Less
Submitted 3 May, 2025;
originally announced May 2025.
-
Momentum-based Distributed Resource Scheduling Optimization Subject to Sector-Bound Nonlinearity and Latency
Authors:
Mohammadreza Doostmohammadian,
Zulfiya R. Gabidullina,
Hamid R. Rabiee
Abstract:
This paper proposes an accelerated consensus-based distributed iterative algorithm for resource allocation and scheduling. The proposed gradient-tracking algorithm introduces an auxiliary variable to add momentum towards the optimal state. We prove that this solution is all-time feasible, implying that the coupling constraint always holds along the algorithm iterative procedure; therefore, the alg…
▽ More
This paper proposes an accelerated consensus-based distributed iterative algorithm for resource allocation and scheduling. The proposed gradient-tracking algorithm introduces an auxiliary variable to add momentum towards the optimal state. We prove that this solution is all-time feasible, implying that the coupling constraint always holds along the algorithm iterative procedure; therefore, the algorithm can be terminated at any time. This is in contrast to the ADMM-based solutions that meet constraint feasibility asymptotically. Further, we show that the proposed algorithm can handle possible link nonlinearity due to logarithmically-quantized data transmission (or any sign-preserving odd sector-bound nonlinear mapping). We prove convergence over uniformly-connected dynamic networks (i.e., a hybrid setup) that may occur in mobile and time-varying multi-agent networks. Further, the latency issue over the network is addressed by proposing delay-tolerant solutions. To our best knowledge, accelerated momentum-based convergence, nonlinear linking, all-time feasibility, uniform network connectivity, and handling (possible) time delays are not altogether addressed in the literature. These contributions make our solution practical in many real-world applications.
△ Less
Submitted 8 March, 2025;
originally announced March 2025.
-
Distributed Observer Design for Tracking Platoon of Connected and Autonomous Vehicles
Authors:
Mohammadreza Doostmohammadian,
Hamid R. Rabiee
Abstract:
Intelligent transportation systems (ITS) aim to advance innovative strategies relating to different modes of transport, traffic management, and autonomous vehicles. This paper studies the platoon of connected and autonomous vehicles (CAV) and proposes a distributed observer to track the state of the CAV dynamics. First, we model the CAV dynamics via an LTI interconnected system. Then, a consensus-…
▽ More
Intelligent transportation systems (ITS) aim to advance innovative strategies relating to different modes of transport, traffic management, and autonomous vehicles. This paper studies the platoon of connected and autonomous vehicles (CAV) and proposes a distributed observer to track the state of the CAV dynamics. First, we model the CAV dynamics via an LTI interconnected system. Then, a consensus-based strategy is proposed to infer the state of the CAV dynamics based on local information exchange over the communication network of vehicles. A linear-matrix-inequality (LMI) technique is adopted for the block-diagonal observer gain design such that this gain is associated in a distributed way and locally to every vehicle. The distributed observer error dynamics is then shown to follow the structure of the Kronecker matrix product of the system dynamics and the adjacency matrix of the CAV network. The notions of survivable network design and redundant observer scheme are further discussed in the paper to address resilience to link and node failure. Finally, we verify our theoretical contributions via numerical simulations.
△ Less
Submitted 30 January, 2025;
originally announced January 2025.
-
Fully Distributed and Quantized Algorithm for MPC-based Autonomous Vehicle Platooning Optimization
Authors:
Mohammadreza Doostmohammadian,
Alireza Aghasi,
Hamid R. Rabiee
Abstract:
Intelligent transportation systems have recently emerged to address the growing interest for safer, more efficient, and sustainable transportation solutions. In this direction, this paper presents distributed algorithms for control and optimization over vehicular networks. First, we formulate the autonomous vehicle platooning framework based on model-predictive-control (MPC) strategies and present…
▽ More
Intelligent transportation systems have recently emerged to address the growing interest for safer, more efficient, and sustainable transportation solutions. In this direction, this paper presents distributed algorithms for control and optimization over vehicular networks. First, we formulate the autonomous vehicle platooning framework based on model-predictive-control (MPC) strategies and present its objective optimization as a cooperative quadratic cost function. Then, we propose a distributed algorithm to locally optimize this objective at every vehicle subject to data quantization over the communication network of vehicles. In contrast to most existing literature that assumes ideal communication channels, log-scale data quantization over the network is addressed in this work, which is more realistic and practical. In particular, we show by simulation that the proposed log-quantized algorithm reaches optimal convergence with less residual and optimality gap. This outperforms the existing literature considering uniform quantization which leads to a large optimality gap and residual.
△ Less
Submitted 30 January, 2025;
originally announced January 2025.
-
Distributed Target Tracking based on Localization with Linear Time-Difference-of-Arrival Measurements: A Delay-Tolerant Networked Estimation Approach
Authors:
Mohammadreza Doostmohammadian,
Themistoklis Charalambous
Abstract:
This paper considers target tracking based on a beacon signal's time-difference-of-arrival (TDOA) to a group of cooperating sensors. The sensors receive a reflected signal from the target where the time-of-arrival (TOA) renders the distance information. The existing approaches include: (i) classic centralized solutions which gather and process the target data at a central unit, (ii) distributed so…
▽ More
This paper considers target tracking based on a beacon signal's time-difference-of-arrival (TDOA) to a group of cooperating sensors. The sensors receive a reflected signal from the target where the time-of-arrival (TOA) renders the distance information. The existing approaches include: (i) classic centralized solutions which gather and process the target data at a central unit, (ii) distributed solutions which assume that the target data is observable in the dense neighborhood of each sensor (to be filtered locally), and (iii) double time-scale distributed methods with high rates of communication/consensus over the network. This work, in order to reduce the network connectivity in (i)-(ii) and communication rate in (iii), proposes a distributed single time-scale technique, which can also handle heterogeneous constant data-exchange delays over the static sensor network. This work assumes only distributed observability (in contrast to local observability in some existing works categorized in (ii)), i.e., the target is observable globally over a (strongly) connected network. The (strong) connectivity further allows for survivable network and $q$-redundant observer design. Each sensor locally shares information and processes the received data in its immediate neighborhood via local linear-matrix-inequalities (LMI) feedback gains to ensure tracking error stability. The same gain matrix works in the presence of heterogeneous delays with no need of redesigning algorithms. Since most existing distributed estimation scenarios are linear (based on consensus), many works use linearization of the existing nonlinear TDOA measurement models where the output matrix is a function of the target position.
△ Less
Submitted 22 December, 2024;
originally announced December 2024.
-
Decentralized Mobile Target Tracking Using Consensus-Based Estimation with Nearly-Constant-Velocity Modeling
Authors:
Amir Ahmad Ghods,
Mohammadreza Doostmohammadian
Abstract:
Mobile target tracking is crucial in various applications such as surveillance and autonomous navigation. This study presents a decentralized tracking framework utilizing a Consensus-Based Estimation Filter (CBEF) integrated with the Nearly-Constant-Velocity (NCV) model to predict a moving target's state. The framework facilitates agents in a network to collaboratively estimate the target's positi…
▽ More
Mobile target tracking is crucial in various applications such as surveillance and autonomous navigation. This study presents a decentralized tracking framework utilizing a Consensus-Based Estimation Filter (CBEF) integrated with the Nearly-Constant-Velocity (NCV) model to predict a moving target's state. The framework facilitates agents in a network to collaboratively estimate the target's position by sharing local observations and achieving consensus despite communication constraints and measurement noise. A saturation-based filtering technique is employed to enhance robustness by mitigating the impact of noisy sensor data. Simulation results demonstrate that the proposed method effectively reduces the Mean Squared Estimation Error (MSEE) over time, indicating improved estimation accuracy and reliability. The findings underscore the effectiveness of the CBEF in decentralized environments, highlighting its scalability and resilience in the presence of uncertainties.
△ Less
Submitted 4 December, 2024;
originally announced December 2024.
-
Logarithmically Quantized Distributed Optimization over Dynamic Multi-Agent Networks
Authors:
Mohammadreza Doostmohammadian,
Sérgio Pequito
Abstract:
Distributed optimization finds many applications in machine learning, signal processing, and control systems. In these real-world applications, the constraints of communication networks, particularly limited bandwidth, necessitate implementing quantization techniques. In this paper, we propose distributed optimization dynamics over multi-agent networks subject to logarithmically quantized data tra…
▽ More
Distributed optimization finds many applications in machine learning, signal processing, and control systems. In these real-world applications, the constraints of communication networks, particularly limited bandwidth, necessitate implementing quantization techniques. In this paper, we propose distributed optimization dynamics over multi-agent networks subject to logarithmically quantized data transmission. Under this condition, data exchange benefits from representing smaller values with more bits and larger values with fewer bits. As compared to uniform quantization, this allows for higher precision in representing near-optimal values and more accuracy of the distributed optimization algorithm. The proposed optimization dynamics comprise a primary state variable converging to the optimizer and an auxiliary variable tracking the objective function's gradient. Our setting accommodates dynamic network topologies, resulting in a hybrid system requiring convergence analysis using matrix perturbation theory and eigenspectrum analysis.
△ Less
Submitted 27 October, 2024;
originally announced October 2024.
-
Nonlinear Perturbation-based Non-Convex Optimization over Time-Varying Networks
Authors:
Mohammadreza Doostmohammadian,
Zulfiya R. Gabidullina,
Hamid R. Rabiee
Abstract:
Decentralized optimization strategies are helpful for various applications, from networked estimation to distributed machine learning. This paper studies finite-sum minimization problems described over a network of nodes and proposes a computationally efficient algorithm that solves distributed convex problems and optimally finds the solution to locally non-convex objective functions. In contrast…
▽ More
Decentralized optimization strategies are helpful for various applications, from networked estimation to distributed machine learning. This paper studies finite-sum minimization problems described over a network of nodes and proposes a computationally efficient algorithm that solves distributed convex problems and optimally finds the solution to locally non-convex objective functions. In contrast to batch gradient optimization in some literature, our algorithm is on a single-time scale with no extra inner consensus loop. It evaluates one gradient entry per node per time. Further, the algorithm addresses link-level nonlinearity representing, for example, logarithmic quantization of the exchanged data or clipping of the exchanged data bits. Leveraging perturbation-based theory and algebraic Laplacian network analysis proves optimal convergence and dynamics stability over time-varying and switching networks. The time-varying network setup might be due to packet drops or link failures. Despite the nonlinear nature of the dynamics, we prove exact convergence in the face of odd sign-preserving sector-bound nonlinear data transmission over the links. Illustrative numerical simulations further highlight our contributions.
△ Less
Submitted 5 August, 2024;
originally announced August 2024.
-
How Clustering Affects the Convergence of Decentralized Optimization over Networks: A Monte-Carlo-based Approach
Authors:
Mohammadreza Doostmohammadian,
Shahaboddin Kharazmi,
Hamid R. Rabiee
Abstract:
Decentralized algorithms have gained substantial interest owing to advancements in cloud computing, Internet of Things (IoT), intelligent transportation networks, and parallel processing over sensor networks. The convergence of such algorithms is directly related to specific properties of the underlying network topology. Specifically, the clustering coefficient is known to affect, for example, the…
▽ More
Decentralized algorithms have gained substantial interest owing to advancements in cloud computing, Internet of Things (IoT), intelligent transportation networks, and parallel processing over sensor networks. The convergence of such algorithms is directly related to specific properties of the underlying network topology. Specifically, the clustering coefficient is known to affect, for example, the controllability/observability and the epidemic growth over networks. In this work, we study the effects of the clustering coefficient on the convergence rate of networked optimization approaches. In this regard, we model the structure of large-scale distributed systems by random scale-free (SF) and clustered scale-free (CSF) networks and compare the convergence rate by tuning the network clustering coefficient. This is done by keeping other relevant network properties (such as power-law degree distribution, number of links, and average degree) unchanged. Monte-Carlo-based simulations are used to compare the convergence rate over many trials of SF graph topologies. Furthermore, to study the convergence rate over real case studies, we compare the clustering coefficient of some real-world networks with the eigenspectrum of the underlying network (as a measure of convergence rate). The results interestingly show higher convergence rate over low-clustered networks. This is significant as one can improve the learning rate of many existing decentralized machine-learning scenarios by tuning the network clustering.
△ Less
Submitted 1 July, 2024;
originally announced July 2024.
-
Log-Scale Quantization in Distributed First-Order Methods: Gradient-based Learning from Distributed Data
Authors:
Mohammadreza Doostmohammadian,
Muhammad I. Qureshi,
Mohammad Hossein Khalesi,
Hamid R. Rabiee,
Usman A. Khan
Abstract:
Decentralized strategies are of interest for learning from large-scale data over networks. This paper studies learning over a network of geographically distributed nodes/agents subject to quantization. Each node possesses a private local cost function, collectively contributing to a global cost function, which the considered methodology aims to minimize. In contrast to many existing papers, the in…
▽ More
Decentralized strategies are of interest for learning from large-scale data over networks. This paper studies learning over a network of geographically distributed nodes/agents subject to quantization. Each node possesses a private local cost function, collectively contributing to a global cost function, which the considered methodology aims to minimize. In contrast to many existing papers, the information exchange among nodes is log-quantized to address limited network-bandwidth in practical situations. We consider a first-order computationally efficient distributed optimization algorithm (with no extra inner consensus loop) that leverages node-level gradient correction based on local data and network-level gradient aggregation only over nearby nodes. This method only requires balanced networks with no need for stochastic weight design. It can handle log-scale quantized data exchange over possibly time-varying and switching network setups. We study convergence over both structured networks (for example, training over data-centers) and ad-hoc multi-agent networks (for example, training over dynamic robotic networks). Through experimental validation, we show that (i) structured networks generally result in a smaller optimality gap, and (ii) log-scale quantization leads to a smaller optimality gap compared to uniform quantization.
△ Less
Submitted 10 January, 2025; v1 submitted 2 June, 2024;
originally announced June 2024.
-
Survey of Distributed Algorithms for Resource Allocation over Multi-Agent Systems
Authors:
Mohammadreza Doostmohammadian,
Alireza Aghasi,
Mohammad Pirani,
Ehsan Nekouei,
Houman Zarrabi,
Reza Keypour,
Apostolos I. Rikos,
Karl H. Johansson
Abstract:
Resource allocation and scheduling in multi-agent systems present challenges due to complex interactions and decentralization. This survey paper provides a comprehensive analysis of distributed algorithms for addressing the distributed resource allocation (DRA) problem over multi-agent systems. It covers a significant area of research at the intersection of optimization, multi-agent systems, and d…
▽ More
Resource allocation and scheduling in multi-agent systems present challenges due to complex interactions and decentralization. This survey paper provides a comprehensive analysis of distributed algorithms for addressing the distributed resource allocation (DRA) problem over multi-agent systems. It covers a significant area of research at the intersection of optimization, multi-agent systems, and distributed consensus-based computing. The paper begins by presenting a mathematical formulation of the DRA problem, establishing a solid foundation for further exploration. Real-world applications of DRA in various domains are examined to underscore the importance of efficient resource allocation, and relevant distributed optimization formulations are presented. The survey then delves into existing solutions for DRA, encompassing linear, nonlinear, primal-based, and dual-formulation-based approaches. Furthermore, this paper evaluates the features and properties of DRA algorithms, addressing key aspects such as feasibility, convergence rate, and network reliability. The analysis of mathematical foundations, diverse applications, existing solutions, and algorithmic properties contributes to a broader comprehension of the challenges and potential solutions for this domain.
△ Less
Submitted 28 January, 2024;
originally announced January 2024.
-
Accelerated Distributed Allocation
Authors:
Mohammadreza Doostmohammadian,
Alireza Aghasi
Abstract:
Distributed allocation finds applications in many scenarios including CPU scheduling, distributed energy resource management, and networked coverage control. In this paper, we propose a fast convergent optimization algorithm with a tunable rate using the signum function. The convergence rate of the proposed algorithm can be managed by changing two parameters. We prove convergence over uniformly-co…
▽ More
Distributed allocation finds applications in many scenarios including CPU scheduling, distributed energy resource management, and networked coverage control. In this paper, we propose a fast convergent optimization algorithm with a tunable rate using the signum function. The convergence rate of the proposed algorithm can be managed by changing two parameters. We prove convergence over uniformly-connected multi-agent networks. Therefore, the solution converges even if the network loses connectivity at some finite time intervals. The proposed algorithm is all-time feasible, implying that at any termination time of the algorithm, the resource-demand feasibility holds. This is in contrast to asymptotic feasibility in many dual formulation solutions (e.g., ADMM) that meet resource-demand feasibility over time and asymptotically.
△ Less
Submitted 28 January, 2024;
originally announced January 2024.
-
Robust-to-Noise Algorithms for Distributed Resource Allocation and Scheduling
Authors:
Mohammadreza Doostmohammadian,
Alireza Aghasi
Abstract:
Efficient resource allocation and scheduling algorithms are essential for various distributed applications, ranging from wireless networks and cloud computing platforms to autonomous multi-agent systems and swarm robotic networks. However, real-world environments are often plagued by uncertainties and noise, leading to sub-optimal performance and increased vulnerability of traditional algorithms.…
▽ More
Efficient resource allocation and scheduling algorithms are essential for various distributed applications, ranging from wireless networks and cloud computing platforms to autonomous multi-agent systems and swarm robotic networks. However, real-world environments are often plagued by uncertainties and noise, leading to sub-optimal performance and increased vulnerability of traditional algorithms. This paper addresses the challenge of robust resource allocation and scheduling in the presence of noise and disturbances. The proposed study introduces a novel sign-based dynamics for developing robust-to-noise algorithms distributed over a multi-agent network that can adaptively handle external disturbances. Leveraging concepts from convex optimization theory, control theory, and network science the framework establishes a principled approach to design algorithms that can maintain key properties such as resource-demand balance and constraint feasibility. Meanwhile, notions of uniform-connectivity and versatile networking conditions are also addressed.
△ Less
Submitted 30 November, 2023;
originally announced November 2023.
-
Discretized Distributed Optimization over Dynamic Digraphs
Authors:
Mohammadreza Doostmohammadian,
Wei Jiang,
Muwahida Liaquat,
Alireza Aghasi,
Houman Zarrabi
Abstract:
We consider a discrete-time model of continuous-time distributed optimization over dynamic directed-graphs (digraphs) with applications to distributed learning. Our optimization algorithm works over general strongly connected dynamic networks under switching topologies, e.g., in mobile multi-agent systems and volatile networks due to link failures. Compared to many existing lines of work, there is…
▽ More
We consider a discrete-time model of continuous-time distributed optimization over dynamic directed-graphs (digraphs) with applications to distributed learning. Our optimization algorithm works over general strongly connected dynamic networks under switching topologies, e.g., in mobile multi-agent systems and volatile networks due to link failures. Compared to many existing lines of work, there is no need for bi-stochastic weight designs on the links. The existing literature mostly needs the link weights to be stochastic using specific weight-design algorithms needed both at the initialization and at all times when the topology of the network changes. This paper eliminates the need for such algorithms and paves the way for distributed optimization over time-varying digraphs. We derive the bound on the gradient-tracking step-size and discrete time-step for convergence and prove dynamic stability using arguments from consensus algorithms, matrix perturbation theory, and Lyapunov theory. This work, particularly, is an improvement over existing stochastic-weight undirected networks in case of link removal or packet drops. This is because the existing literature may need to rerun time-consuming and computationally complex algorithms for stochastic design, while the proposed strategy works as long as the underlying network is weight-symmetric and balanced. The proposed optimization framework finds applications to distributed classification and learning.
△ Less
Submitted 26 March, 2024; v1 submitted 14 November, 2023;
originally announced November 2023.
-
Distributed Delay-Tolerant Strategies for Equality-Constraint Sum-Preserving Resource Allocation
Authors:
Mohammadreza Doostmohammadian,
Alireza Aghasi,
Maria Vrakopoulou,
Hamid R. Rabiee,
Usman A. Khan,
Themistoklis Charalambou
Abstract:
This paper proposes two nonlinear dynamics to solve constrained distributed optimization problem for resource allocation over a multi-agent network. In this setup, coupling constraint refers to resource-demand balance which is preserved at all-times. The proposed solutions can address various model nonlinearities, for example, due to quantization and/or saturation. Further, it allows to reach fast…
▽ More
This paper proposes two nonlinear dynamics to solve constrained distributed optimization problem for resource allocation over a multi-agent network. In this setup, coupling constraint refers to resource-demand balance which is preserved at all-times. The proposed solutions can address various model nonlinearities, for example, due to quantization and/or saturation. Further, it allows to reach faster convergence or to robustify the solution against impulsive noise or uncertainties. We prove convergence over weakly connected networks using convex analysis and Lyapunov theory. Our findings show that convergence can be reached for general sign-preserving odd nonlinearity. We further propose delay-tolerant mechanisms to handle general bounded heterogeneous time-varying delays over the communication network of agents while preserving all-time feasibility. This work finds application in CPU scheduling and coverage control among others. This paper advances the state-of-the-art by addressing (i) possible nonlinearity on the agents/links, meanwhile handling (ii) resource-demand feasibility at all times, (iii) uniform-connectivity instead of all-time connectivity, and (iv) possible heterogeneous and time-varying delays. To our best knowledge, no existing work addresses contributions (i)-(iv) altogether. Simulations and comparative analysis are provided to corroborate our contributions.
△ Less
Submitted 27 October, 2023;
originally announced October 2023.
-
D-SVM over Networked Systems with Non-Ideal Linking Conditions
Authors:
Mohammadreza Doostmohammadian,
Alireza Aghasi,
Houman Zarrabi
Abstract:
This paper considers distributed optimization algorithms, with application in binary classification via distributed support-vector-machines (D-SVM) over multi-agent networks subject to some link nonlinearities. The agents solve a consensus-constraint distributed optimization cooperatively via continuous-time dynamics, while the links are subject to strongly sign-preserving odd nonlinear conditions…
▽ More
This paper considers distributed optimization algorithms, with application in binary classification via distributed support-vector-machines (D-SVM) over multi-agent networks subject to some link nonlinearities. The agents solve a consensus-constraint distributed optimization cooperatively via continuous-time dynamics, while the links are subject to strongly sign-preserving odd nonlinear conditions. Logarithmic quantization and clipping (saturation) are two examples of such nonlinearities. In contrast to existing literature that mostly considers ideal links and perfect information exchange over linear channels, we show how general sector-bounded models affect the convergence to the optimizer (i.e., the SVM classifier) over dynamic balanced directed networks. In general, any odd sector-bounded nonlinear mapping can be applied to our dynamics. The main challenge is to show that the proposed system dynamics always have one zero eigenvalue (associated with the consensus) and the other eigenvalues all have negative real parts. This is done by recalling arguments from matrix perturbation theory. Then, the solution is shown to converge to the agreement state under certain conditions. For example, the gradient tracking (GT) step size is tighter than the linear case by factors related to the upper/lower sector bounds. To the best of our knowledge, no existing work in distributed optimization and learning literature considers non-ideal link conditions.
△ Less
Submitted 13 April, 2023;
originally announced April 2023.
-
Consensus-based Networked Tracking in Presence of Heterogeneous Time-Delays
Authors:
Mohammadreza Doostmohammadian,
Mohammad Pirani,
Usman A. Khan
Abstract:
We propose a distributed (single) target tracking scheme based on networked estimation and consensus algorithms over static sensor networks. The tracking part is based on linear time-difference-of-arrival (TDOA) measurement proposed in our previous works. This paper, in particular, develops delay-tolerant distributed filtering solutions over sparse data-transmission networks. We assume general arb…
▽ More
We propose a distributed (single) target tracking scheme based on networked estimation and consensus algorithms over static sensor networks. The tracking part is based on linear time-difference-of-arrival (TDOA) measurement proposed in our previous works. This paper, in particular, develops delay-tolerant distributed filtering solutions over sparse data-transmission networks. We assume general arbitrary heterogeneous delays at different links. This may occur in many realistic large-scale applications where the data-sharing between different nodes is subject to latency due to communication-resource constraints or large spatially distributed sensor networks. The solution we propose in this work shows improved performance (verified by both theory and simulations) in such scenarios. Another privilege of such distributed schemes is the possibility to add localized fault-detection and isolation (FDI) strategies along with survivable graph-theoretic design, which opens many follow-up venues to this research. To our best knowledge no such delay-tolerant distributed linear algorithm is given in the existing distributed tracking literature.
△ Less
Submitted 15 February, 2023;
originally announced February 2023.
-
Distributed Constraint-Coupled Optimization over Lossy Networks
Authors:
Mohammadreza Doostmohammadian,
Usman A. Khan,
Alireza Aghasi,
Themistoklis Charalambous
Abstract:
This paper considers distributed resource allocation and sum-preserving constrained optimization over lossy networks, where the links are unreliable and subject to packet drops. We define the conditions to ensure convergence under packet drops and link removal by focusing on two main properties of our allocation algorithm: (i) The weight-stochastic condition in typical consensus schemes is reduced…
▽ More
This paper considers distributed resource allocation and sum-preserving constrained optimization over lossy networks, where the links are unreliable and subject to packet drops. We define the conditions to ensure convergence under packet drops and link removal by focusing on two main properties of our allocation algorithm: (i) The weight-stochastic condition in typical consensus schemes is reduced to balanced weights, with no need for readjusting the weights to satisfy stochasticity. (ii) The algorithm does not require all-time connectivity but instead uniform connectivity over some non-overlapping finite time intervals. First, we prove that our algorithm provides primal-feasible allocation at every iteration step and converges under the conditions (i)-(ii) and some other mild conditions on the nonlinear iterative dynamics. These nonlinearities address possible practical constraints in real applications due to, for example, saturation or quantization among others. Then, using (i)-(ii) and the notion of bond-percolation theory, we relate the packet drop rate and the network percolation threshold to the (finite) number of iterations ensuring uniform connectivity and, thus, convergence towards the optimum value.
△ Less
Submitted 30 August, 2022;
originally announced August 2022.
-
DTAC-ADMM: Delay-Tolerant Augmented Consensus ADMM-based Algorithm for Distributed Resource Allocation
Authors:
Mohammadreza Doostmohammadian,
Wei Jiang,
Themistoklis Charalambous
Abstract:
Latency is inherent in almost all real-world networked applications. In this paper, we propose a distributed allocation strategy over multi-agent networks with delayed communications. The state of each agent (or node) represents its share of assigned resources out of a fixed amount (equal to overall demand). Every node locally updates its state toward optimizing a global allocation cost function v…
▽ More
Latency is inherent in almost all real-world networked applications. In this paper, we propose a distributed allocation strategy over multi-agent networks with delayed communications. The state of each agent (or node) represents its share of assigned resources out of a fixed amount (equal to overall demand). Every node locally updates its state toward optimizing a global allocation cost function via received information of its neighbouring nodes even when the data exchange over the network is heterogeneously delayed at different links. The update is based on the alternating direction method of multipliers (ADMM) formulation subject to both sum-preserving coupling-constraint and local box-constraints. The solution is derivative-free and holds for general (not necessarily differentiable) convex cost models. We use the notion of augmented consensus over undirected networks to model delayed information exchange and for convergence analysis. We simulate our \textit{delay-tolerant} algorithm for
△ Less
Submitted 30 August, 2022;
originally announced August 2022.
-
Distributed CPU Scheduling Subject to Nonlinear Constraints
Authors:
Mohammadreza Doostmohammadian,
Alireza Aghasi,
Apostolos I. Rikos,
Andreas Grammenos,
Evangelia Kalyvianaki,
Christoforos N. Hadjicostis,
Karl H. Johansson,
Themistoklis Charalambous
Abstract:
This paper considers a network of collaborating agents for local resource allocation subject to nonlinear model constraints. In many applications, it is required (or desirable) that the solution be anytime feasible in terms of satisfying the sum-preserving global constraint. Motivated by this, sufficient conditions on the nonlinear mapping for anytime feasibility (or non-asymptotic feasibility) ar…
▽ More
This paper considers a network of collaborating agents for local resource allocation subject to nonlinear model constraints. In many applications, it is required (or desirable) that the solution be anytime feasible in terms of satisfying the sum-preserving global constraint. Motivated by this, sufficient conditions on the nonlinear mapping for anytime feasibility (or non-asymptotic feasibility) are addressed in this paper. For the two proposed distributed solutions, one converges over directed weight-balanced networks and the other one over undirected networks. In particular, we elaborate on uniform quantization and discuss the notion of ε-accurate solution, which gives an estimate of how close we can get to the exact optimizer subject to different quantization levels. This work, further, handles general (possibly non-quadratic) strictly convex objective functions with application to CPU allocation among a cloud of data centers via distributed solutions. The results can be used as a coordination mechanism to optimally balance the tasks and CPU resources among a group of networked servers while addressing quantization or limited server capacity.
Index Terms: multi-agent systems, sum-preserving resource allocation, distributed optimization, anytime feasibility
△ Less
Submitted 30 August, 2022;
originally announced August 2022.
-
Linear TDOA-based Measurements for Distributed Estimation and Localized Tracking
Authors:
Mohammadreza Doostmohammadian,
Themistoklis Charalambous
Abstract:
We propose a linear time-difference-of-arrival (TDOA) measurement model to improve \textit{distributed} estimation performance for localized target tracking. We design distributed filters over sparse (possibly large-scale) communication networks using consensus-based data-fusion techniques. The proposed distributed and localized tracking protocols considerably reduce the sensor network's required…
▽ More
We propose a linear time-difference-of-arrival (TDOA) measurement model to improve \textit{distributed} estimation performance for localized target tracking. We design distributed filters over sparse (possibly large-scale) communication networks using consensus-based data-fusion techniques. The proposed distributed and localized tracking protocols considerably reduce the sensor network's required connectivity and communication rate. We, further, consider $κ$-redundant observability and fault-tolerant design in case of losing communication links or sensor nodes. We present the minimal conditions on the remaining sensor network (after link/node removal) such that the distributed observability is still preserved and, thus, the sensor network can track the (single) maneuvering target. The motivation is to reduce the communication load versus the processing load, as the computational units are, in general, less costly than the communication devices. We evaluate the tracking performance via simulations in MATLAB.
△ Less
Submitted 26 April, 2022;
originally announced April 2022.
-
Distributed Anomaly Detection and Estimation over Sensor Networks: Observational-Equivalence and Q-Redundant Observer Design
Authors:
Mohammadreza Doostmohammadian,
Themistoklis Charalambous
Abstract:
In this paper, we study stateless and stateful physics-based anomaly detection scenarios via distributed estimation over sensor networks. In the stateful case, the detector keeps track of the sensor residuals (i.e., the difference of estimated and true outputs) and reports an alarm if certain statistics of the recorded residuals deviate over a predefined threshold, e.g., χ^2 (Chi-square) detector.…
▽ More
In this paper, we study stateless and stateful physics-based anomaly detection scenarios via distributed estimation over sensor networks. In the stateful case, the detector keeps track of the sensor residuals (i.e., the difference of estimated and true outputs) and reports an alarm if certain statistics of the recorded residuals deviate over a predefined threshold, e.g., χ^2 (Chi-square) detector. Instead, only instantaneous deviation of the residuals raises the alarm in the stateless case without considering the history of the sensor outputs and estimation data. Given (approximate) false-alarm rate for both cases, we propose a probabilistic threshold design based on the noise statistics. We show by simulation that increasing the window length in the stateful case may not necessarily reduce the false-alarm rate. On the other hand, it adds unwanted delay to raise the alarm. The distributed aspect of the proposed detection algorithm enables local isolation of the faulty sensors with possible recovery solutions by adding redundant observationally-equivalent sensors. We, then, offer a mechanism to design Q-redundant distributed observers, robust to failure (or removal) of up to Q sensors over the network.
△ Less
Submitted 4 April, 2022;
originally announced April 2022.
-
Distributed Finite-Sum Constrained Optimization subject to Nonlinearity on the Node Dynamics
Authors:
Mohammadreza Doostmohammadian,
Maria Vrakopoulou,
Alireza Aghasi,
Themistoklis Charalambous
Abstract:
Motivated by recent development in networking and parallel data-processing, we consider a distributed and localized finite-sum (or fixed-sum) allocation technique to solve resource-constrained convex optimization problems over multi-agent networks (MANs). Such networks include (smart) agents representing an intelligent entity capable of communication, processing, and decision-making. In particular…
▽ More
Motivated by recent development in networking and parallel data-processing, we consider a distributed and localized finite-sum (or fixed-sum) allocation technique to solve resource-constrained convex optimization problems over multi-agent networks (MANs). Such networks include (smart) agents representing an intelligent entity capable of communication, processing, and decision-making. In particular, we consider problems subject to practical nonlinear constraints on the dynamics of the agents in terms of their communications and actuation capabilities (referred to as the node dynamics), e.g., networks of mobile robots subject to actuator saturation and quantized communication. The considered distributed sum-preserving optimization solution further enables adding purposeful nonlinear constraints, for example, sign-based nonlinearities, to reach convergence in predefined-time or robust to impulsive noise and disturbances in faulty environments. Moreover, convergence can be achieved under minimal network connectivity requirements among the agents; thus, the solution is applicable over dynamic networks where the channels come and go due to the agent's mobility and limited range. This paper discusses how various nonlinearity constraints on the optimization problem (e.g., collaborative allocation of resources) can be addressed for different applications via a distributed setup (over a network).
△ Less
Submitted 28 March, 2022;
originally announced March 2022.
-
1st-Order Dynamics on Nonlinear Agents for Resource Allocation over Uniformly-Connected Networks
Authors:
Mohammadreza Doostmohammadian,
Alireza Aghasi,
Maria Vrakopoulou,
Themistoklis Charalambous
Abstract:
A general nonlinear $1$st-order consensus-based solution for distributed constrained convex optimization is proposed with network resource allocation applications. The solution is used to optimize continuously-differentiable strictly convex cost functions over weakly-connected undirected networks, while it is anytime feasible and models various nonlinearities to account for imperfections and const…
▽ More
A general nonlinear $1$st-order consensus-based solution for distributed constrained convex optimization is proposed with network resource allocation applications. The solution is used to optimize continuously-differentiable strictly convex cost functions over weakly-connected undirected networks, while it is anytime feasible and models various nonlinearities to account for imperfections and constraints on the (physical model of) agents in terms of limited actuation capabilities, e.g., quantization and saturation. Due to such inherent nonlinearities, the existing linear solutions considering ideal agent models may not necessarily converge with guaranteed optimality and anytime feasibility. Some applications also impose specific nonlinearities, e.g., convergence in fixed/finite-time or sign-based robust disturbance-tolerant dynamics. Our proposed distributed protocol generalizes such nonlinear models. Putting convex set analysis together with nonsmooth Lyapunov analysis, we prove convergence, (i) regardless of the particular type of nonlinearity, and (ii) with weak network-connectivity requirements (uniform-connectivity).
△ Less
Submitted 19 November, 2021; v1 submitted 10 September, 2021;
originally announced September 2021.
-
Analysis of Contractions in System Graphs: Application to State Estimation
Authors:
Mohammadreza Doostmohammadian,
Themistoklis Charalambous,
Miadreza Shafie-khah,
Hamid R. Rabiee,
Usman A. Khan
Abstract:
Observability and estimation are closely tied to the system structure, which can be visualized as a system graph--a graph that captures the inter-dependencies within the state variables. For example, in social system graphs such inter-dependencies represent the social interactions of different individuals. It was recently shown that contractions, a key concept from graph theory, in the system grap…
▽ More
Observability and estimation are closely tied to the system structure, which can be visualized as a system graph--a graph that captures the inter-dependencies within the state variables. For example, in social system graphs such inter-dependencies represent the social interactions of different individuals. It was recently shown that contractions, a key concept from graph theory, in the system graph are critical to system observability, as (at least) one state measurement in every contraction is necessary for observability. Thus, the size and number of contractions are critical in recovering for loss of observability. In this paper, the correlation between the average-size/number of contractions and the global clustering coefficient (GCC) of the system graph is studied. Our empirical results show that estimating systems with high GCC requires fewer measurements, and in case of measurement failure, there are fewer possible options to find substitute measurement that recovers the system's observability. This is significant as by tuning the GCC, we can improve the observability properties of large-scale engineered networks, such as social networks and smart grid.
△ Less
Submitted 22 May, 2021;
originally announced May 2021.
-
Simultaneous Distributed Estimation and Attack Detection/Isolation in Social Networks: Structural Observability, Kronecker-Product Network, and Chi-Square Detector
Authors:
Mohammadreza Doostmohammadian,
Themistoklis Charalambous,
Miadreza Shafie-khah,
Nader Meskin,
Usman A. Khan
Abstract:
This paper considers distributed estimation of linear systems when the state observations are corrupted with Gaussian noise of unbounded support and under possible random adversarial attacks. We consider sensors equipped with single time-scale estimators and local chi-square ($χ^2$) detectors to simultaneously opserve the states, share information, fuse the noise/attack-corrupted data locally, and…
▽ More
This paper considers distributed estimation of linear systems when the state observations are corrupted with Gaussian noise of unbounded support and under possible random adversarial attacks. We consider sensors equipped with single time-scale estimators and local chi-square ($χ^2$) detectors to simultaneously opserve the states, share information, fuse the noise/attack-corrupted data locally, and detect possible anomalies in their own observations. While this scheme is applicable to a wide variety of systems associated with full-rank (invertible) matrices, we discuss it within the context of distributed inference in social networks. The proposed technique outperforms existing results in the sense that: (i) we consider Gaussian noise with no simplifying upper-bound assumption on the support; (ii) all existing $χ^2$-based techniques are centralized while our proposed technique is distributed, where the sensors \textit{locally} detect attacks, with no central coordinator, using specific probabilistic thresholds; and (iii) no local-observability assumption at a sensor is made, which makes our method feasible for large-scale social networks. Moreover, we consider a Linear Matrix Inequalities (LMI) approach to design block-diagonal gain (estimator) matrices under appropriate constraints for isolating the attacks.
△ Less
Submitted 22 May, 2021;
originally announced May 2021.
-
Distributed support-vector-machine over dynamic balanced directed networks
Authors:
Mohammadreza Doostmohammadian,
Alireza Aghasi,
Themistoklis Charalambous,
Usman A. Khan
Abstract:
In this paper, we consider the binary classification problem via distributed Support-Vector-Machines (SVM), where the idea is to train a network of agents, with limited share of data, to cooperatively learn the SVM classifier for the global database. Agents only share processed information regarding the classifier parameters and the gradient of the local loss functions instead of their raw data. I…
▽ More
In this paper, we consider the binary classification problem via distributed Support-Vector-Machines (SVM), where the idea is to train a network of agents, with limited share of data, to cooperatively learn the SVM classifier for the global database. Agents only share processed information regarding the classifier parameters and the gradient of the local loss functions instead of their raw data. In contrast to the existing work, we propose a continuous-time algorithm that incorporates network topology changes in discrete jumps. This hybrid nature allows us to remove chattering that arises because of the discretization of the underlying CT process. We show that the proposed algorithm converges to the SVM classifier over time-varying weight balanced directed graphs by using arguments from the matrix perturbation theory.
△ Less
Submitted 1 April, 2021;
originally announced April 2021.
-
Consensus-Based Distributed Estimation in the Presence of Heterogeneous, Time-Invariant Delays
Authors:
Mohammadreza Doostmohammadian,
Usman A. Khan,
Mohammad Pirani,
Themistoklis Charalambous
Abstract:
Classical distributed estimation scenarios typically assume timely and reliable exchanges of information over the sensor network. This paper, in contrast, considers single time-scale distributed estimation via a sensor network subject to transmission time-delays. The proposed discrete-time networked estimator consists of two steps: (i) consensus on (delayed) a-priori estimates, and (ii) measuremen…
▽ More
Classical distributed estimation scenarios typically assume timely and reliable exchanges of information over the sensor network. This paper, in contrast, considers single time-scale distributed estimation via a sensor network subject to transmission time-delays. The proposed discrete-time networked estimator consists of two steps: (i) consensus on (delayed) a-priori estimates, and (ii) measurement update. The sensors only share their a-priori estimates with their out-neighbors over (possibly) time-delayed transmission links. The delays are assumed to be fixed over time, heterogeneous, and known. We assume distributed observability instead of local observability, which significantly reduces the communication/sensing loads on sensors. Using the notions of augmented matrices and Kronecker product, the convergence of the proposed estimator over strongly-connected networks is proved for a specific upper-bound on the time-delay.
△ Less
Submitted 7 September, 2021; v1 submitted 1 April, 2021;
originally announced April 2021.
-
Measurement partitioning and observational equivalence in state estimation
Authors:
Mohammadreza Doostmohammadian,
Usman A. Khan
Abstract:
This letter studies measurement partitioning and equivalence in state estimation based on graph-theoretic principles. We show that a set of critical measurements (required to ensure LTI state-space observability) can be further partitioned into two types:~$α$ and~$β$. This partitioning is driven by different graphical (or algebraic) methods used to define the corresponding measurements. Subsequent…
▽ More
This letter studies measurement partitioning and equivalence in state estimation based on graph-theoretic principles. We show that a set of critical measurements (required to ensure LTI state-space observability) can be further partitioned into two types:~$α$ and~$β$. This partitioning is driven by different graphical (or algebraic) methods used to define the corresponding measurements. Subsequently, we describe observational equivalence, i.e. given an~$α$ (or~$β$) measurement, say~$y_i$, what is the set of measurements equivalent to~$y_i$, such that only one measurement in this set is required to ensure observability? Since~$α$ and~$β$ measurements are cast using different algebraic and graphical characteristics, their equivalence sets are also derived using different algebraic and graph-theoretic principles. We illustrate the related concepts on an appropriate system digraph.
△ Less
Submitted 16 December, 2014;
originally announced December 2014.
