Showing 1–50 of 50 results for author: Kia, S

ResQue Greedy: Rewiring Sequential Greedy for Improved Submodular Maximization

Authors: Joan Vendrell Gallart, Alan Kuhnle, Solmaz Kia

Abstract: This paper introduces Rewired Sequential Greedy (ResQue Greedy), an enhanced approach for submodular maximization under cardinality constraints. By integrating a novel set curvature metric within a lattice-based framework, ResQue Greedy identifies and corrects suboptimal decisions made by the standard sequential greedy algorithm. Specifically, a curvature-aware rewiring strategy is employed to dyn… ▽ More This paper introduces Rewired Sequential Greedy (ResQue Greedy), an enhanced approach for submodular maximization under cardinality constraints. By integrating a novel set curvature metric within a lattice-based framework, ResQue Greedy identifies and corrects suboptimal decisions made by the standard sequential greedy algorithm. Specifically, a curvature-aware rewiring strategy is employed to dynamically redirect the solution path, leading to improved approximation performance over the conventional sequential greedy algorithm without significantly increasing computational complexity. Numerical experiments demonstrate that ResQue Greedy achieves tighter near-optimality bounds compared to the traditional sequential greedy method. △ Less

Submitted 19 May, 2025; originally announced May 2025.

NavEX: A Multi-Agent Coverage in Non-Convex and Uneven Environments via Exemplar-Clustering

Authors: Donipolo Ghimire, Carlos Nieto-Granda, Solmaz S. Kia

Abstract: This paper addresses multi-agent deployment in non-convex and uneven environments. To overcome the limitations of traditional approaches, we introduce Navigable Exemplar-Based Dispatch Coverage (NavEX), a novel dispatch coverage framework that combines exemplar-clustering with obstacle-aware and traversability-aware shortest distances, offering a deployment framework based on submodular optimizati… ▽ More This paper addresses multi-agent deployment in non-convex and uneven environments. To overcome the limitations of traditional approaches, we introduce Navigable Exemplar-Based Dispatch Coverage (NavEX), a novel dispatch coverage framework that combines exemplar-clustering with obstacle-aware and traversability-aware shortest distances, offering a deployment framework based on submodular optimization. NavEX provides a unified approach to solve two critical coverage tasks: (a) fair-access deployment, aiming to provide equitable service by minimizing agent-target distances, and (b) hotspot deployment, prioritizing high-density target regions. A key feature of NavEX is the use of exemplar-clustering for the coverage utility measure, which provides the flexibility to employ non-Euclidean distance metrics that do not necessarily conform to the triangle inequality. This allows NavEX to incorporate visibility graphs for shortest-path computation in environments with planar obstacles, and traversability-aware RRT* for complex, rugged terrains. By leveraging submodular optimization, the NavEX framework enables efficient, near-optimal solutions with provable performance guarantees for multi-agent deployment in realistic and complex settings, as demonstrated by our simulations. △ Less

Submitted 29 April, 2025; originally announced April 2025.

Submodular Maximization Subject to Uniform and Partition Matroids: From Theory to Practical Applications and Distributed Solutions

Authors: Solmaz S. Kia

Abstract: This article provides a comprehensive exploration of submodular maximization problems, focusing on those subject to uniform and partition matroids. Crucial for a wide array of applications in fields ranging from computer science to systems engineering, submodular maximization entails selecting elements from a discrete set to optimize a submodular utility function under certain constraints. We expl… ▽ More This article provides a comprehensive exploration of submodular maximization problems, focusing on those subject to uniform and partition matroids. Crucial for a wide array of applications in fields ranging from computer science to systems engineering, submodular maximization entails selecting elements from a discrete set to optimize a submodular utility function under certain constraints. We explore the foundational aspects of submodular functions and matroids, outlining their core properties and illustrating their application through various optimization scenarios. Central to our exposition is the discussion on algorithmic strategies, particularly the sequential greedy algorithm and its efficacy under matroid constraints. Additionally, we extend our analysis to distributed submodular maximization, highlighting the challenges and solutions for large-scale, distributed optimization problems. This work aims to succinctly bridge the gap between theoretical insights and practical applications in submodular maximization, providing a solid foundation for researchers navigating this intricate domain. △ Less

Submitted 2 January, 2025; originally announced January 2025.

Comments: 22 pages, 4 figures, 71 references

Time-Varying Convex Optimization with $O(n)$ Computational Complexity

Authors: M. Rostami, S. S. Kia

Abstract: In this article, we consider the problem of unconstrained time-varying convex optimization, where the cost function changes with time. We provide an in-depth technical analysis of the problem and argue why freezing the cost at each time step and taking finite steps toward the minimizer is not the best tracking solution for this problem. We propose a set of algorithms that by taking into account th… ▽ More In this article, we consider the problem of unconstrained time-varying convex optimization, where the cost function changes with time. We provide an in-depth technical analysis of the problem and argue why freezing the cost at each time step and taking finite steps toward the minimizer is not the best tracking solution for this problem. We propose a set of algorithms that by taking into account the temporal variation of the cost aim to reduce the tracking error of the time-varying minimizer of the problem. The main contribution of our work is that our proposed algorithms only require the first-order derivatives of the cost function with respect to the decision variable. This approach significantly reduces computational cost compared to the existing algorithms, which use the inverse of the Hessian of the cost. Specifically, the proposed algorithms reduce the computational cost from $O(n^3)$ to $O(n)$ per timestep, where $n$ is the size of the decision variable. Avoiding the inverse of the Hessian also makes our algorithms applicable to non-convex optimization problems. We refer to these algorithms as $O(n)$-algorithms. These $O(n)$-algorithms are designed to solve the problem for different scenarios based on the available temporal information about the cost. We illustrate our results through various examples, including the solution of a model predictive control problem framed as a convex optimization problem with a streaming time-varying cost function. △ Less

Submitted 24 October, 2024; v1 submitted 19 October, 2024; originally announced October 2024.

Comments: arXiv admin note: text overlap with arXiv:2310.07925

FORWARD: Feasibility Oriented Random-Walk Inspired Algorithm for Radial Reconfiguration in Distribution Networks

Authors: Joan Vendrell, Russell Bent, Solmaz Kia

Abstract: We consider an optimal flow distribution problem in which the goal is to find a radial configuration that minimizes resistance-induced quadratic distribution costs while ensuring delivery of inputs from multiple sources to all sinks to meet their demands. This problem has critical applications in various distribution systems, such as electricity, where efficient energy flow is crucial for both eco… ▽ More We consider an optimal flow distribution problem in which the goal is to find a radial configuration that minimizes resistance-induced quadratic distribution costs while ensuring delivery of inputs from multiple sources to all sinks to meet their demands. This problem has critical applications in various distribution systems, such as electricity, where efficient energy flow is crucial for both economic and environmental reasons. Due to its complexity, finding an optimal solution is computationally challenging and NP-hard. In this paper, we propose a novel algorithm called FORWARD, which leverages graph theory to efficiently identify feasible configurations in polynomial time. By drawing parallels with random walk processes on electricity networks, our method simplifies the search space, significantly reducing computational effort while maintaining performance. The FORWARD algorithm employs a combination of network preprocessing, intelligent partitioning, and strategic sampling to construct radial configurations that meet flow requirements, finding a feasible solution in polynomial time. Numerical experiments demonstrate the effectiveness of our approach, highlighting its potential for real-world applications in optimizing distribution networks. △ Less

Submitted 17 October, 2024; originally announced October 2024.

FedScalar: A Communication efficient Federated Learning

Authors: M. Rostami, S. S. Kia

Abstract: Federated learning (FL) has gained considerable popularity for distributed machine learning due to its ability to preserve the privacy of participating agents by eliminating the need for data aggregation. Nevertheless, communication costs between agents and the central server in FL are substantial in large-scale problems and remain a limiting factor for this algorithm. This paper introduces an inn… ▽ More Federated learning (FL) has gained considerable popularity for distributed machine learning due to its ability to preserve the privacy of participating agents by eliminating the need for data aggregation. Nevertheless, communication costs between agents and the central server in FL are substantial in large-scale problems and remain a limiting factor for this algorithm. This paper introduces an innovative algorithm, called \emph{FedScalar}, within the federated learning framework aimed at improving communication efficiency. Unlike traditional FL methods that require agents to send high-dimensional vectors to the server, \emph{FedScalar} enables agents to communicate updates using a single scalar. Each agent encodes its updated model parameters into a scalar through the inner product between its local update difference and a random vector, which is then transmitted to the server. The server decodes this information by projecting the averaged scalar values onto the random vector. Our method thereby significantly reduces communication overhead. Technically, we demonstrate that the proposed algorithm achieves a convergence rate of $O(1/\sqrt{K})$ to a stationary point for smooth, non-convex loss functions. Additionally, our analysis shows that altering the underlying distribution of the random vector generated by the server can reduce the variance during the aggregation step of the algorithm. Finally, we validate the performance and communication efficiency of our algorithm with numerical simulations. △ Less

Submitted 3 October, 2024; originally announced October 2024.

Optimality Gap of Decentralized Submodular Maximization under Probabilistic Communication

Authors: Joan Vendrell, Solmaz Kia

Abstract: This paper considers the problem of decentralized submodular maximization subject to partition matroid constraint using a sequential greedy algorithm with probabilistic inter-agent message-passing. We propose a communication-aware framework where the probability of successful communication between connected devices is considered. Our analysis introduces the notion of the probabilistic optimality g… ▽ More This paper considers the problem of decentralized submodular maximization subject to partition matroid constraint using a sequential greedy algorithm with probabilistic inter-agent message-passing. We propose a communication-aware framework where the probability of successful communication between connected devices is considered. Our analysis introduces the notion of the probabilistic optimality gap, highlighting its potential influence on determining the message-passing sequence based on the agent's broadcast reliability and strategic decisions regarding agents that can broadcast their messages multiple times in a resource-limited environment. This work not only contributes theoretical insights but also has practical implications for designing and analyzing decentralized systems in uncertain communication environments. A numerical example demonstrates the impact of our results. △ Less

Submitted 16 September, 2024; originally announced September 2024.

Multi-agent Coverage Control: From Discrete Assignments to Continuous Multi-agent Distribution Matching

Authors: Solmaz Kia, Sonia Martinez

Abstract: The multi-agent spatial coverage control problem encompasses a broad research domain, dealing with both dynamic and static deployment strategies, discrete-task assignments, and spatial distribution-matching deployment. Coverage control may involve the deployment of a finite number of agents or a continuum through centralized or decentralized, locally-interacting schemes. All these problems can be… ▽ More The multi-agent spatial coverage control problem encompasses a broad research domain, dealing with both dynamic and static deployment strategies, discrete-task assignments, and spatial distribution-matching deployment. Coverage control may involve the deployment of a finite number of agents or a continuum through centralized or decentralized, locally-interacting schemes. All these problems can be solved via a different taxonomy of deployment algorithms for multiple agents. Depending on the application scenario, these problems involve from purely discrete descriptions of tasks (finite loads) and agents (finite resources), to a mixture of discrete and continuous elements, to fully continuous descriptions of the same. Yet, it is possible to find common features that underline all the above formulations, which we aim to illustrate here. By doing so, we aim to point the reader to novel references related to these problems. The short article outline is the following: Static coverage via concurrent area partitioning and assignment; Static coverage as a discrete task assignment; and Continuum task assignment for large-scale swarms. △ Less

Submitted 18 July, 2024; originally announced July 2024.

Journal ref: Summer 2024 DSCD Newsletter

Sequential Gaussian Variational Inference for Nonlinear State Estimation and Its Application in Robot Navigation

Authors: Min-Won Seo, Solmaz S. Kia

Abstract: Probabilistic state estimation is essential for robots navigating uncertain environments. Accurately and efficiently managing uncertainty in estimated states is key to robust robotic operation. However, nonlinearities in robotic platforms pose significant challenges that require advanced estimation techniques. Gaussian variational inference (GVI) offers an optimization perspective on the estimatio… ▽ More Probabilistic state estimation is essential for robots navigating uncertain environments. Accurately and efficiently managing uncertainty in estimated states is key to robust robotic operation. However, nonlinearities in robotic platforms pose significant challenges that require advanced estimation techniques. Gaussian variational inference (GVI) offers an optimization perspective on the estimation problem, providing analytically tractable solutions and efficiencies derived from the geometry of Gaussian space. We propose a Sequential Gaussian Variational Inference (S-GVI) method to address nonlinearity and provide efficient sequential inference processes. Our approach integrates sequential Bayesian principles into the GVI framework, which are addressed using statistical approximations and gradient updates on the information geometry. Validations through simulations and real-world experiments demonstrate significant improvements in state estimation over the Maximum A Posteriori (MAP) estimation method. △ Less

Submitted 18 November, 2024; v1 submitted 7 July, 2024; originally announced July 2024.

Comments: 8 pages

Projected Forward Gradient-Guided Frank-Wolfe Algorithm via Variance Reduction

Authors: M. Rostami, S. S. Kia

Abstract: This paper aims to enhance the use of the Frank-Wolfe (FW) algorithm for training deep neural networks. Similar to any gradient-based optimization algorithm, FW suffers from high computational and memory costs when computing gradients for DNNs. This paper introduces the application of the recently proposed projected forward gradient (Projected-FG) method to the FW framework, offering reduced compu… ▽ More This paper aims to enhance the use of the Frank-Wolfe (FW) algorithm for training deep neural networks. Similar to any gradient-based optimization algorithm, FW suffers from high computational and memory costs when computing gradients for DNNs. This paper introduces the application of the recently proposed projected forward gradient (Projected-FG) method to the FW framework, offering reduced computational cost similar to backpropagation and low memory utilization akin to forward propagation. Our results show that trivial application of the Projected-FG introduces non-vanishing convergence error due to the stochastic noise that the Projected-FG method introduces in the process. This noise results in an non-vanishing variance in the Projected-FG estimated gradient. To address this, we propose a variance reduction approach by aggregating historical Projected-FG directions. We demonstrate rigorously that this approach ensures convergence to the optimal solution for convex functions and to a stationary point for non-convex functions. These convergence properties are validated through a numerical example, showcasing the approach's effectiveness and efficiency. △ Less

Submitted 25 December, 2024; v1 submitted 19 March, 2024; originally announced March 2024.

Stein-MAP: A Sequential Variational Inference Framework for Maximum A Posteriori Estimation

Authors: Min-Won Seo, Solmaz S. Kia

Abstract: State estimation poses substantial challenges in robotics, often involving encounters with multimodality in real-world scenarios. To address these challenges, it is essential to calculate Maximum a posteriori (MAP) sequences from joint probability distributions of latent states and observations over time. However, it generally involves a trade-off between approximation errors and computational com… ▽ More State estimation poses substantial challenges in robotics, often involving encounters with multimodality in real-world scenarios. To address these challenges, it is essential to calculate Maximum a posteriori (MAP) sequences from joint probability distributions of latent states and observations over time. However, it generally involves a trade-off between approximation errors and computational complexity. In this article, we propose a new method for MAP sequence estimation called Stein-MAP, which effectively manages multimodality with fewer approximation errors while significantly reducing computational and memory burdens. Our key contribution lies in the introduction of a sequential variational inference framework designed to handle temporal dependencies among transition states within dynamical system models. The framework integrates Stein's identity from probability theory and reproducing kernel Hilbert space (RKHS) theory, enabling computationally efficient MAP sequence estimation. As a MAP sequence estimator, Stein-MAP boasts a computational complexity of O(N), where N is the number of particles, in contrast to the O(N^2) complexity of the Viterbi algorithm. The proposed method is empirically validated through real-world experiments focused on range-only (wireless) localization. The results demonstrate a substantial enhancement in state estimation compared to existing methods. A remarkable feature of Stein-MAP is that it can attain improved state estimation with only 40 to 50 particles, as opposed to the 1000 particles that the particle filter or its variants require. △ Less

Submitted 16 December, 2023; v1 submitted 14 December, 2023; originally announced December 2023.

Comments: 13 pages

Stein Coverage: a Variational Inference Approach to Distribution-matching Multisensor Deployment

Authors: Donipolo Ghimire, Solmaz S. Kia

Abstract: This paper examines the spatial coverage optimization problem for multiple sensors in a known convex environment, where the coverage service of each sensor is heterogeneous and anisotropic. We introduce the Stein Coverage algorithm, a distribution-matching coverage approach that aims to place sensors at positions and orientations such that their collective coverage distribution is as close as poss… ▽ More This paper examines the spatial coverage optimization problem for multiple sensors in a known convex environment, where the coverage service of each sensor is heterogeneous and anisotropic. We introduce the Stein Coverage algorithm, a distribution-matching coverage approach that aims to place sensors at positions and orientations such that their collective coverage distribution is as close as possible to the event distribution. To select the most important representative points from the coverage event distribution, Stein Coverage utilizes the Stein Variational Gradient Descent (SVGD), a deterministic sampling method from the variational inference literature. An innovation in our work is the introduction of a repulsive force between the samples in the SVGD algorithm to spread the samples and avoid footprint overlap for the deployed sensors. After pinpointing the points of interest for deployment, Stein Coverage solves the multisensor assignment problem using a bipartite optimal matching process. Simulations demonstrate the advantages of the Stein Coverage method compared to conventional Voronoi partitioning multisensor deployment methods. △ Less

Submitted 12 December, 2023; originally announced December 2023.

First-Order Dynamic Optimization for Streaming Convex Costs

Authors: M. Rostami, H. Moradian, S. S. Kia

Abstract: This paper proposes a set of novel optimization algorithms for solving a class of convex optimization problems with time-varying streaming cost function. We develop an approach to track the optimal solution with a bounded error. Unlike the existing results, our algorithm is executed only by using the first-order derivatives of the cost function which makes it computationally efficient for optimiza… ▽ More This paper proposes a set of novel optimization algorithms for solving a class of convex optimization problems with time-varying streaming cost function. We develop an approach to track the optimal solution with a bounded error. Unlike the existing results, our algorithm is executed only by using the first-order derivatives of the cost function which makes it computationally efficient for optimization with time-varying cost function. We compare our algorithms to the gradient descent algorithm and show why gradient descent is not an effective solution for optimization problems with time-varying cost. Several examples including solving a model predictive control problem cast as a convex optimization problem with a streaming time-varying cost function demonstrate our results. △ Less

Submitted 11 October, 2023; originally announced October 2023.

Bayesian Online Learning for Human-assisted Target Localization

Authors: Min-Won Seo, Solmaz S. Kia

Abstract: We consider a human-assisted autonomy sensor fusion for dynamic target localization in a Bayesian framework. Autonomous sensor-based tracking systems can suffer from observability and target detection failure. Humans possess valuable qualitative information derived from their past knowledge and rapid situational awareness that can give them an advantage over machine perception in many scenarios. T… ▽ More We consider a human-assisted autonomy sensor fusion for dynamic target localization in a Bayesian framework. Autonomous sensor-based tracking systems can suffer from observability and target detection failure. Humans possess valuable qualitative information derived from their past knowledge and rapid situational awareness that can give them an advantage over machine perception in many scenarios. To compensate for the shortcomings of an autonomous tracking system, we propose to collect spatial sensing information from human operators who visually monitor the target and can provide target localization information in the form of free sketches encircling the area where the target is located. However, human inputs cannot be taken deterministically and trusted absolutely due to their inherent subjectivity and variability. Our focus in this paper is to construct an adaptive probabilistic model for human-provided inputs where the adaptation terms capture the level of reliability of the human inputs. The next contribution of this paper is a novel joint Bayesian learning method to fuse human and autonomous sensor inputs in a manner that the dynamic changes in human detection reliability are also captured and accounted for. Unlike deep learning frameworks, a unique aspect of this Bayesian modeling framework is its analytical closed-form update equations. This feature provides computational efficiency and allows for online learning from limited data sets. Simulations demonstrate our results, underscoring the value of human-machine collaboration in autonomous systems. △ Less

Submitted 4 October, 2024; v1 submitted 22 August, 2023; originally announced August 2023.

Comments: 7 figures

Distributed Unconstrained Optimization with Time-varying Cost Functions

Authors: Amir-Salar Esteki, Solmaz S. Kia

Abstract: In this paper, we propose a novel solution for the distributed unconstrained optimization problem where the total cost is the summation of time-varying local cost functions of a group networked agents. The objective is to track the optimal trajectory that minimizes the total cost at each time instant. Our approach consists of a two-stage dynamics, where the first one samples the first and second d… ▽ More In this paper, we propose a novel solution for the distributed unconstrained optimization problem where the total cost is the summation of time-varying local cost functions of a group networked agents. The objective is to track the optimal trajectory that minimizes the total cost at each time instant. Our approach consists of a two-stage dynamics, where the first one samples the first and second derivatives of the local costs periodically to construct an estimate of the descent direction towards the optimal trajectory, and the second one uses this estimate and a consensus term to drive local states towards the time-varying solution while reaching consensus. The first part is carried out by the implementation of a weighted average consensus algorithm in the discrete-time framework and the second part is performed with a continuous-time dynamics. Using the Lyapunov stability analysis, an upper bound on the gradient of the total cost is obtained which is asymptotically reached. This bound is characterized by the properties of the local costs. To demonstrate the performance of the proposed method, a numerical example is conducted that studies tuning the algorithm's parameters and their effects on the convergence of local states to the optimal trajectory. △ Less

Submitted 12 December, 2022; originally announced December 2022.

Fast model averaging via buffered states and first-order accelerated optimization algorithms

Authors: Amir-Salar Esteki, Hossein Moradian, Solmaz S. Kia

Abstract: In this letter, we study the problem of accelerating reaching average consensus over connected graphs in a discrete-time communication setting. Literature has shown that consensus algorithms can be accelerated by increasing the graph connectivity or optimizing the weights agents place on the information received from their neighbors. Here, instead of altering the communication graph, we investigat… ▽ More In this letter, we study the problem of accelerating reaching average consensus over connected graphs in a discrete-time communication setting. Literature has shown that consensus algorithms can be accelerated by increasing the graph connectivity or optimizing the weights agents place on the information received from their neighbors. Here, instead of altering the communication graph, we investigate two methods that use buffered states to accelerate reaching average consensus over a given graph. In the first method, we study how convergence rate of the well-known first-order Laplacian average consensus algorithm changes when agreement feedback is generated from buffered states. For this study, we obtain a sufficient condition on the ranges of buffered state that leads to faster convergence. In the second proposed method, we show how the average consensus problem can be cast as a convex optimization problem and solved by first-order accelerated optimization algorithms for strongly-convex cost functions. We construct an accelerated average consensus algorithm using the so-called Triple Momentum optimization algorithm. The first approach requires less global knowledge for choosing the step size, whereas the second one converges faster in our numerical results by using extra information from the graph topology. We demonstrate our results by implementing the proposed algorithms in a Gaussian Mixture Model (GMM) estimation problem used in sensor networks. △ Less

Submitted 10 November, 2022; originally announced November 2022.

Comments: arXiv admin note: substantial text overlap with arXiv:2206.09916

Federated Learning Using Variance Reduced Stochastic Gradient for Probabilistically Activated Agents

Authors: M. R. Rostami, S. S. Kia

Abstract: This paper proposes an algorithm for Federated Learning (FL) with a two-layer structure that achieves both variance reduction and a faster convergence rate to an optimal solution in the setting where each agent has an arbitrary probability of selection in each iteration. In distributed machine learning, when privacy matters, FL is a functional tool. Placing FL in an environment where it has some i… ▽ More This paper proposes an algorithm for Federated Learning (FL) with a two-layer structure that achieves both variance reduction and a faster convergence rate to an optimal solution in the setting where each agent has an arbitrary probability of selection in each iteration. In distributed machine learning, when privacy matters, FL is a functional tool. Placing FL in an environment where it has some irregular connections of agents (devices), reaching a trained model in both an economical and quick way can be a demanding job. The first layer of our algorithm corresponds to the model parameter propagation across agents done by the server. In the second layer, each agent does its local update with a stochastic and variance-reduced technique called Stochastic Variance Reduced Gradient (SVRG). We leverage the concept of variance reduction from stochastic optimization when the agents want to do their local update step to reduce the variance caused by stochastic gradient descent (SGD). We provide a convergence bound for our algorithm which improves the rate from $O(\frac{1}{\sqrt{K}})$ to $O(\frac{1}{K})$ by using a constant step-size. We demonstrate the performance of our algorithm using numerical examples. △ Less

Submitted 1 April, 2023; v1 submitted 25 October, 2022; originally announced October 2022.

The fastest linearly converging discrete-time average consensus using buffered information

Authors: Amir-Salar Esteki, Hossein Moradian, Solmaz S. Kia

Abstract: In this letter, we study the problem of accelerating reaching average consensus over connected graphs in a discrete-time communication setting. Literature has shown that consensus algorithms can be accelerated by increasing the graph connectivity or optimizing the weights agents place on the information received from their neighbors. In this letter instead of altering the communication graph, we i… ▽ More In this letter, we study the problem of accelerating reaching average consensus over connected graphs in a discrete-time communication setting. Literature has shown that consensus algorithms can be accelerated by increasing the graph connectivity or optimizing the weights agents place on the information received from their neighbors. In this letter instead of altering the communication graph, we investigate two methods that use buffered states to accelerate reaching average consensus over a given graph. In the first method, we study how convergence rate of the well-known first-order Laplacian average consensus algorithm changes with delayed feedback and obtain a sufficient condition on the ranges of delay that leads to faster convergence. In the second proposed method, we show how average consensus problem can be cast as a convex optimization problem and solved by first-order accelerated optimization algorithms for strongly-convex cost functions. We construct the fastest converging average consensus algorithm using the so-called Triple Momentum optimization algorithm. We demonstrate our results using an in-network linear regression problem, which is formulated as two average consensus problems. △ Less

Submitted 20 June, 2022; originally announced June 2022.

Comments: 6 pages, Europe Control Conference

PROMISSING: Pruning Missing Values in Neural Networks

Authors: Seyed Mostafa Kia, Nastaran Mohammadian Rad, Daniel van Opstal, Bart van Schie, Andre F. Marquand, Josien Pluim, Wiepke Cahn, Hugo G. Schnack

Abstract: While data are the primary fuel for machine learning models, they often suffer from missing values, especially when collected in real-world scenarios. However, many off-the-shelf machine learning models, including artificial neural network models, are unable to handle these missing values directly. Therefore, extra data preprocessing and curation steps, such as data imputation, are inevitable befo… ▽ More While data are the primary fuel for machine learning models, they often suffer from missing values, especially when collected in real-world scenarios. However, many off-the-shelf machine learning models, including artificial neural network models, are unable to handle these missing values directly. Therefore, extra data preprocessing and curation steps, such as data imputation, are inevitable before learning and prediction processes. In this study, we propose a simple and intuitive yet effective method for pruning missing values (PROMISSING) during learning and inference steps in neural networks. In this method, there is no need to remove or impute the missing values; instead, the missing values are treated as a new source of information (representing what we do not know). Our experiments on simulated data, several classification and regression benchmarks, and a multi-modal clinical dataset show that PROMISSING results in similar prediction performance compared to various imputation techniques. In addition, our experiments show models trained using PROMISSING techniques are becoming less decisive in their predictions when facing incomplete samples with many unknowns. This finding hopefully advances machine learning models from being pure predicting machines to more realistic thinkers that can also say "I do not know" when facing incomplete sources of information. △ Less

Submitted 3 June, 2022; originally announced June 2022.

Online Target Localization using Adaptive Belief Propagation in the HMM Framework

Authors: Min-Won Seo, Solmaz S. Kia

Abstract: This paper proposes a novel adaptive sample space-based Viterbi algorithm for target localization in an online manner. The method relies on discretizing the target's motion space into cells representing a finite number of hidden states. Then, the most probable trajectory of the tracked target is computed via dynamic programming in a Hidden Markov Model (HMM) framework. The proposed method uses a B… ▽ More This paper proposes a novel adaptive sample space-based Viterbi algorithm for target localization in an online manner. The method relies on discretizing the target's motion space into cells representing a finite number of hidden states. Then, the most probable trajectory of the tracked target is computed via dynamic programming in a Hidden Markov Model (HMM) framework. The proposed method uses a Bayesian estimation framework which is neither limited to Gaussian noise models nor requires a linearized target motion model or sensor measurement models. However, an HMM-based approach to localization can suffer from poor computational complexity in scenarios where the number of hidden states increases due to high-resolution modeling or target localization in a large space. To improve this poor computational complexity, this paper proposes a belief propagation in the most probable belief space with a low to high-resolution sequentially, reducing the required resources significantly. The proposed method is inspired by the k-d Tree algorithm (e.g., quadtree) commonly used in the computer vision field. Experimental tests using an ultra-wideband (UWB) sensor network demonstrate our results. △ Less

Submitted 15 August, 2022; v1 submitted 7 March, 2022; originally announced March 2022.

Comments: 8 pages, 8 figures

Journal ref: IEEE Robotics and Automation Letters. Vol 7 Issue 4 (2022) 10288 - 10295

Learning Contraction Policies from Offline Data

Authors: Navid Rezazadeh, Maxwell Kolarich, Solmaz S. Kia, Negar Mehr

Abstract: This paper proposes a data-driven method for learning convergent control policies from offline data using Contraction theory. Contraction theory enables constructing a policy that makes the closed-loop system trajectories inherently convergent towards a unique trajectory. At the technical level, identifying the contraction metric, which is the distance metric with respect to which a robot's trajec… ▽ More This paper proposes a data-driven method for learning convergent control policies from offline data using Contraction theory. Contraction theory enables constructing a policy that makes the closed-loop system trajectories inherently convergent towards a unique trajectory. At the technical level, identifying the contraction metric, which is the distance metric with respect to which a robot's trajectories exhibit contraction is often non-trivial. We propose to jointly learn the control policy and its corresponding contraction metric while enforcing contraction. To achieve this, we learn an implicit dynamics model of the robotic system from an offline data set consisting of the robot's state and input trajectories. Using this learned dynamics model, we propose a data augmentation algorithm for learning contraction policies. We randomly generate samples in the state-space and propagate them forward in time through the learned dynamics model to generate auxiliary sample trajectories. We then learn both the control policy and the contraction metric such that the distance between the trajectories from the offline data set and our generated auxiliary sample trajectories decreases over time. We evaluate the performance of our proposed framework on simulated robotic goal-reaching tasks and demonstrate that enforcing contraction results in faster convergence and greater robustness of the learned policy. △ Less

Submitted 3 February, 2022; v1 submitted 10 December, 2021; originally announced December 2021.

Learning-based Measurement Scheduling for Loosely-Coupled Cooperative Localization

Authors: Jianan Zhu, Solmaz S. Kia

Abstract: In cooperative localization, communicating mobile agents use inter-agent relative measurements to improve their dead-reckoning-based global localization. Measurement scheduling enables an agent to decide which subset of available inter-agent relative measurements it should process when its computational resources are limited. Optimal measurement scheduling is an NP-hard combinatorial optimization… ▽ More In cooperative localization, communicating mobile agents use inter-agent relative measurements to improve their dead-reckoning-based global localization. Measurement scheduling enables an agent to decide which subset of available inter-agent relative measurements it should process when its computational resources are limited. Optimal measurement scheduling is an NP-hard combinatorial optimization problem. The so-called sequential greedy (SG) algorithm is a popular suboptimal polynomial-time solution for this problem. However, the merit function evaluation for the SG algorithms requires access to the state estimate vector and error covariance matrix of all the landmark agents (teammates that an agent can take measurements from). This paper proposes a measurement scheduling for CL that follows the SG approach but reduces the communication and computation cost by using a neural network-based surrogate model as a proxy for the SG algorithm's merit function. The significance of this model is that it is driven by local information and only a scalar metadata from the landmark agents. This solution addresses the time and memory complexity issues of running the SG algorithm in three ways: (a) reducing the inter-agent communication message size, (b) decreasing the complexity of function evaluations by using a simpler surrogate (proxy) function, (c) reducing the required memory size.Simulations demonstrate our results. △ Less

Submitted 6 December, 2021; originally announced December 2021.

Comments: 6 pages, 7 figures

Distributed Strategy Selection: A Submodular Set Function Maximization Approach

Authors: Navid Rezazadeh, Solmaz S. Kia

Abstract: Constrained submodular set function maximization problems often appear in multi-agent decision-making problems with a discrete feasible set. A prominent example is the problem of multi-agent mobile sensor placement over a discrete domain. Submodular set function optimization problems, however, are known to be NP-hard. This paper considers a class of submodular optimization problems that consist of… ▽ More Constrained submodular set function maximization problems often appear in multi-agent decision-making problems with a discrete feasible set. A prominent example is the problem of multi-agent mobile sensor placement over a discrete domain. Submodular set function optimization problems, however, are known to be NP-hard. This paper considers a class of submodular optimization problems that consist of maximization of a monotone and submodular set function subject to a uniform matroid constraint over a group of networked agents that communicate over a connected undirected graph. We work in the value oracle model where the only access of the agents to the utility function is through a black box that returns the utility function value. We propose a distributed suboptimal polynomial-time algorithm that enables each agent to obtain its respective strategy via local interactions with its neighboring agents. Our solution is a fully distributed gradient-based algorithm using the submodular set functions' multilinear extension followed by a distributed stochastic Pipage rounding procedure. This algorithm results in a strategy set that when the team utility function is evaluated at worst case, the utility function value is in 1/c(1-e^(-c)-O(1/T)) of the optimal solution with c to be the curvature of the submodular function. An example demonstrates our results. △ Less

Submitted 29 July, 2021; originally announced July 2021.

Comments: arXiv admin note: text overlap with arXiv:2011.14499

Deterministic Privacy Preservation in Static Average Consensus Problem

Authors: Amir-Salar Esteki, Solmaz S. Kia

Abstract: In this paper we consider the problem of privacy preservation in the static average consensus problem. This problem normally is solved by proposing privacy preservation augmentations for the popular first order Laplacian-based algorithm. These mechanisms however come with computational overhead, may need coordination among the agents to choose their parameters and also alter the transient response… ▽ More In this paper we consider the problem of privacy preservation in the static average consensus problem. This problem normally is solved by proposing privacy preservation augmentations for the popular first order Laplacian-based algorithm. These mechanisms however come with computational overhead, may need coordination among the agents to choose their parameters and also alter the transient response of the algorithm. In this paper we show that an alternative iterative algorithm that is proposed in the literature in the context of dynamic average consensus problem has intrinsic privacy preservation and can be used as a privacy preserving algorithm that yields the same performance behavior as the well-known Laplacian consensus algorithm but without the overheads that come with the existing privacy preservation methods. △ Less

Submitted 7 December, 2020; originally announced December 2020.

Multi-Agent Maximization of a Monotone Submodular Function via Maximum Consensus

Authors: Navid Rezazadeh, Solmaz S. Kia

Abstract: Constrained submodular set function maximization problems often appear in multi-agent decision-making problems with a discrete feasible set. A prominent example is the problem of multi-agent mobile sensor placement over a discrete domain. However, submodular set function optimization problems are known to be NP-hard. In this paper, we consider a class of submodular optimization problems that consi… ▽ More Constrained submodular set function maximization problems often appear in multi-agent decision-making problems with a discrete feasible set. A prominent example is the problem of multi-agent mobile sensor placement over a discrete domain. However, submodular set function optimization problems are known to be NP-hard. In this paper, we consider a class of submodular optimization problems that consists of maximization of a monotone and submodular set function subject to a uniform matroid constraint over a group of networked agents that communicate over a connected undirected graph. Our objective is to obtain a distributed suboptimal polynomial-time algorithm that enables each agent to obtain its respective policy via local interactions with its neighboring agents. Our solution is a fully distributed gradient-based algorithm using the multilinear extension of the submodular set functions and exploiting a maximum consensus scheme. This algorithm results in a policy set that when the team objective function is evaluated at worst case the objective function value is in $1-1/e-O(1/T)$ of the optimal solution. An example demonstrates our results. △ Less

Submitted 29 November, 2020; originally announced November 2020.

An IMM-based Decentralized Cooperative Localization with LoS and NLoS UWB Inter-agent Ranging

Authors: Jianan Zhu, Solmaz S. Kia

Abstract: This paper investigates an infra-structure free global localization of a group of communicating mobile agents (e.g., first responders or exploring robots) via an ultra-wideband (UWB) inter-agent ranging aided dead-reckoning. We propose a loosely coupled cooperative localization algorithm that acts as an augmentation atop the local dead-reckoning system of each mobile agent. This augmentation becom… ▽ More This paper investigates an infra-structure free global localization of a group of communicating mobile agents (e.g., first responders or exploring robots) via an ultra-wideband (UWB) inter-agent ranging aided dead-reckoning. We propose a loosely coupled cooperative localization algorithm that acts as an augmentation atop the local dead-reckoning system of each mobile agent. This augmentation becomes active only when an agent wants to process a relative measurement it has taken. The main contribution of this paper is addressing the challenges in the proper processing of the UWB range measurements in the framework of a loosely coupled cooperative localization. Even though UWB offers a decimeter level accuracy in line-of-sight (LoS) ranging, its accuracy degrades significantly in non-line-of-sight (NLoS) due to the significant unknown positive bias in the measurements. Thus, the measurement models for the UWB LoS and NLoS ranging conditions are different, and proper processing of NLoS measurements requires a bias compensation measure. We also show that, in practice, the measurement modal discriminators determine the type of UWB range measurements should be probabilistic. To take into account the probabilistic nature of the NLoS identifiers when processing UWB inter-agent ranging feedback, we employ an interacting multiple model (IMM) estimator in our localization filter. We also propose a bias compensation method for NLoS UWB measurements. The effectiveness of our cooperative localization is demonstrated via an experiment for a group of pedestrians who use UWB relative range measurements among themselves to improve their shoe-mounted INS geolocation. △ Less

Submitted 30 December, 2020; v1 submitted 8 September, 2020; originally announced September 2020.

Dynamic Active Average Consensus and its Application in Containment Control

Authors: Yi-Fan Chung, Solmaz S. Kia

Abstract: This paper proposes a continuous-time dynamic active weighted average consensus algorithm in which the agents can alternate between active and passive modes depending on their ability to access to their reference input. The objective is to enable all the agents, both active and passive, to track the weighted average of the reference inputs of the active agents. The algorithm is modeled as a switch… ▽ More This paper proposes a continuous-time dynamic active weighted average consensus algorithm in which the agents can alternate between active and passive modes depending on their ability to access to their reference input. The objective is to enable all the agents, both active and passive, to track the weighted average of the reference inputs of the active agents. The algorithm is modeled as a switched linear system whose convergence properties are carefully studied considering the agents' piece-wise constant access to the reference signals and possible piece-wise constant weights of the agents. We also study the discrete-time implementation of this algorithm. Next, we show how a containment control problem, in which a group of followers should track the convex hull of a set of observed leaders, can be cast as an active average consensus problem, and solved efficiently by our proposed dynamic active average consensus algorithm. Numerical examples demonstrate our results. △ Less

Submitted 13 August, 2020; originally announced August 2020.

Gender and Emotion Recognition from Implicit User Behavior Signals

Authors: Maneesh Bilalpur, Seyed Mostafa Kia, Mohan Kankanhalli, Ramanathan Subramanian

Abstract: This work explores the utility of implicit behavioral cues, namely, Electroencephalogram (EEG) signals and eye movements for gender recognition (GR) and emotion recognition (ER) from psychophysical behavior. Specifically, the examined cues are acquired via low-cost, off-the-shelf sensors. 28 users (14 male) recognized emotions from unoccluded (no mask) and partially occluded (eye or mouth masked)… ▽ More This work explores the utility of implicit behavioral cues, namely, Electroencephalogram (EEG) signals and eye movements for gender recognition (GR) and emotion recognition (ER) from psychophysical behavior. Specifically, the examined cues are acquired via low-cost, off-the-shelf sensors. 28 users (14 male) recognized emotions from unoccluded (no mask) and partially occluded (eye or mouth masked) emotive faces; their EEG responses contained gender-specific differences, while their eye movements were characteristic of the perceived facial emotions. Experimental results reveal that (a) reliable GR and ER is achievable with EEG and eye features, (b) differential cognitive processing of negative emotions is observed for females and (c) eye gaze-based gender differences manifest under partial face occlusion, as typified by the eye and mouth mask conditions. △ Less

Submitted 23 June, 2020; originally announced June 2020.

Comments: Under consideration for publication in IEEE Trans. Affective Computing

Hierarchical Bayesian Regression for Multi-Site Normative Modeling of Neuroimaging Data

Authors: Seyed Mostafa Kia, Hester Huijsdens, Richard Dinga, Thomas Wolfers, Maarten Mennes, Ole A. Andreassen, Lars T. Westlye, Christian F. Beckmann, Andre F. Marquand

Abstract: Clinical neuroimaging has recently witnessed explosive growth in data availability which brings studying heterogeneity in clinical cohorts to the spotlight. Normative modeling is an emerging statistical tool for achieving this objective. However, its application remains technically challenging due to difficulties in properly dealing with nuisance variation, for example due to variability in image… ▽ More Clinical neuroimaging has recently witnessed explosive growth in data availability which brings studying heterogeneity in clinical cohorts to the spotlight. Normative modeling is an emerging statistical tool for achieving this objective. However, its application remains technically challenging due to difficulties in properly dealing with nuisance variation, for example due to variability in image acquisition devices. Here, in a fully probabilistic framework, we propose an application of hierarchical Bayesian regression (HBR) for multi-site normative modeling. Our experimental results confirm the superiority of HBR in deriving more accurate normative ranges on large multi-site neuroimaging data compared to widely used methods. This provides the possibility i) to learn the normative range of structural and functional brain measures on large multi-site data; ii) to recalibrate and reuse the learned model on local small data; therefore, HBR closes the technical loop for applying normative modeling as a medical tool for the diagnosis and prognosis of mental disorders. △ Less

Submitted 25 May, 2020; originally announced May 2020.

Comments: To be published in MICCAI 2020 proceedings

Measurement Scheduling for Cooperative Localization in Resource-Constrained Conditions

Authors: Qi Yan, Li Jiang, Solmaz Kia

Abstract: This paper studies the measurement scheduling problem for a group of N mobile robots moving on a flat surface that are preforming cooperative localization (CL). We consider a scenario in which due to the limited on-board resources such as battery life and communication bandwidth only a given number of relative measurements per robot are allowed at observation and update stage. Optimal selection of… ▽ More This paper studies the measurement scheduling problem for a group of N mobile robots moving on a flat surface that are preforming cooperative localization (CL). We consider a scenario in which due to the limited on-board resources such as battery life and communication bandwidth only a given number of relative measurements per robot are allowed at observation and update stage. Optimal selection of which teammates a robot should take a relative measurement from such that the updated joint localization uncertainty of the team is minimized is an NP-hard problem. In this paper, we propose a suboptimal greedy approach that allows each robot to choose its landmark robots locally in polynomial time. Our method, unlike the known results in the literature, does not assume full-observability of CL algorithm. Moreover, it does not require inter-robot communication at scheduling stage. That is, there is no need for the robots to collaborate to carry out the landmark robot selections. We discuss the application of our method in the context of an state-of-the-art decentralized CL algorithm and demonstrate its effectiveness through numerical simulations. Even though our solution does not come with rigorous performance guarantees, its low computational cost along with no communication requirement makes it an appealing solution for operatins with resource constrained robots. △ Less

Submitted 20 January, 2020; v1 submitted 10 December, 2019; originally announced December 2019.

Journal ref: IEEE Robotics and Automation Letters (Volume: 5, Issue: 2, April 2020)

A Study on Accelerating Average Consensus Algorithms Using Delayed Feedback

Authors: Hossein Moradian, Solmaz S. Kia

Abstract: In this paper, we study accelerating a Laplacian-based dynamic average consensus algorithm by splitting the conventional delay-free disagreement feedback into weighted summation of a current and an outdated term. We determine for what weighted sum there exists a range of time delay that results in the higher rate of convergence for the algorithm. For such weights, using the Lambert W function, we… ▽ More In this paper, we study accelerating a Laplacian-based dynamic average consensus algorithm by splitting the conventional delay-free disagreement feedback into weighted summation of a current and an outdated term. We determine for what weighted sum there exists a range of time delay that results in the higher rate of convergence for the algorithm. For such weights, using the Lambert W function, we obtain the rate increasing range of the time delay, the maximum reachable rate and comment on the value of the corresponding maximizer delay. We also study the effect of use of outdated feedback on the control effort of the agents and show that only for some specific affine combination of the immediate and outdated feedback the control effort of the agents does not go beyond that of the delay-free algorithm. Additionally, we demonstrate that using outdated feedback does not increase the steady state tracking error of the average consensus algorithm. Lastly, we determine the optimum combination of the current and the outdated feedback weights to achieve the maximum increase in the rate of convergence without increasing the control effort of the agents. We demonstrate our results through a numerical example. △ Less

Submitted 9 December, 2019; originally announced December 2019.

Distributed Leader Following of an Active Leader for Linear Heterogeneous Multi-Agent Systems

Authors: Yi-Fan Chung, Solmaz S. Kia

Abstract: This paper considers a leader-following problem for a group of heterogeneous linear time invariant (LTI) followers that are interacting over a directed acyclic graph. Only a subset of the followers has access to the state of the leader in specific sampling times. The dynamics of the leader that generates its sampled states is unknown to the followers. For interaction topologies in which the leader… ▽ More This paper considers a leader-following problem for a group of heterogeneous linear time invariant (LTI) followers that are interacting over a directed acyclic graph. Only a subset of the followers has access to the state of the leader in specific sampling times. The dynamics of the leader that generates its sampled states is unknown to the followers. For interaction topologies in which the leader is a global sink in the graph, we propose a distributed algorithm that allows the followers to arrive at the sampled state of the leader by the time the next sample arrives. Our algorithm is a practical solution for a leader-following problem when there is no information available about the state of the leader except its instantaneous value at the sampling times. Our algorithm also allows the followers to track the sampled state of the leader with a locally chosen offset that can be time-varying. When the followers are mobile agents whose state or part of their state is their position vector, the offset mechanism can be used to enable the followers to form a transnational invariant formation about the sampled state of the leader. We prove that the control input of the followers to take them from one sampled state to the next one is minimum energy. We also show in case of the homogeneous followers, after the first sampling epoch the states and inputs of all the followers are synchronized with each other. Numerical examples demonstrate our results. △ Less

Submitted 19 November, 2019; v1 submitted 17 September, 2019; originally announced September 2019.

Cluster-based Distributed Augmented Lagrangian Algorithm for a Class of Constrained Convex Optimization Problems

Authors: Hossein Moradian, Solmaz S. Kia

Abstract: We propose a distributed solution for a constrained convex optimization problem over a network of clustered agents each consisted of a set of subagents. The communication range of the clustered agents is such that they can form a connected undirected graph topology. The total cost in this optimization problem is the sum of the local convex costs of the subagents of each cluster. We seek a minimize… ▽ More We propose a distributed solution for a constrained convex optimization problem over a network of clustered agents each consisted of a set of subagents. The communication range of the clustered agents is such that they can form a connected undirected graph topology. The total cost in this optimization problem is the sum of the local convex costs of the subagents of each cluster. We seek a minimizer of this cost subject to a set of affine equality constraints, and a set of affine inequality constraints specifying the bounds on the decision variables if such bounds exist. We design our distributed algorithm in a cluster-based framework which results in a significant reduction in communication and computation costs. Our proposed distributed solution is a novel continuous-time algorithm that is linked to the augmented Lagrangian approach. It converges asymptotically when the local cost functions are convex and exponentially when they are strongly convex and have Lipschitz gradients. Moreover, we use an $ε$-exact penalty function to address the inequality constraints and derive an explicit lower bound on the penalty function weight to guarantee convergence to $ε$-neighborhood of the global minimum value of the cost. A numerical example demonstrates our results. △ Less

Submitted 2 April, 2021; v1 submitted 19 August, 2019; originally announced August 2019.

A sub-modular receding horizon solution for mobile multi-agent persistent monitoring

Authors: Navid Rezazadeh, Solmaz S. Kia

Abstract: We study the problem of persistent monitoring of a finite number of inter-connected geographical nodes by a group of heterogeneous mobile agents. We assign to each geographical node a concave and increasing reward function that resets to zero after an agent's visit. Then, we design the optimal dispatch policy of which nodes to visit at what time and by what agent by finding a policy set that maxim… ▽ More We study the problem of persistent monitoring of a finite number of inter-connected geographical nodes by a group of heterogeneous mobile agents. We assign to each geographical node a concave and increasing reward function that resets to zero after an agent's visit. Then, we design the optimal dispatch policy of which nodes to visit at what time and by what agent by finding a policy set that maximizes a utility that is defined as the total reward collected at visit times. We show that this optimization problem is NP-hard and its computational complexity increases exponentially with the number of the agents and the length of the mission horizon. By showing that the utility function is a monotone increasing and submodular set function of agents' policy, we proceed to propose a suboptimal dispatch policy design with a known optimality gap. To reduce the time complexity of constructing the feasible search set and also to induce robustness to changes in the operational factors, we perform our suboptimal policy design in a receding horizon fashion. Then, to compensate for the shortsightedness of the receding horizon approach for reward distribution beyond the feasible policies of the agents over the receding horizon, we add a new term to our utility, which provides a measure of nodal importance beyond the receding horizon's sight. This term gives the policy design an intuition to steer the agents towards the nodes with higher rewards on the patrolling graph. Finally, we discuss how our proposed algorithm can be implemented in a decentralized manner. A simulation study demonstrates our results. △ Less

Submitted 21 October, 2020; v1 submitted 12 August, 2019; originally announced August 2019.

Cooperative Localization under Limited Connectivity

Authors: Jianan Zhu, Solmaz S. Kia

Abstract: We report two decentralized multi-agent cooperative localization algorithms in which, to reduce the communication cost, inter-agent state estimate correlations are not maintained but accounted for implicitly. In our first algorithm, to guarantee filter consistency, we account for unknown inter-agent correlations via an upper bound on the joint covariance matrix of the agents. In the second method,… ▽ More We report two decentralized multi-agent cooperative localization algorithms in which, to reduce the communication cost, inter-agent state estimate correlations are not maintained but accounted for implicitly. In our first algorithm, to guarantee filter consistency, we account for unknown inter-agent correlations via an upper bound on the joint covariance matrix of the agents. In the second method, we use an optimization framework to estimate the unknown inter-agent cross-covariance matrix. In our algorithms, each agent localizes itself in a global coordinate frame using a local filter driven by local dead reckoning and occasional absolute measurement updates, and opportunistically corrects its pose estimate whenever it can obtain relative measurements with respect to other mobile agents. To process any relative measurement, only the agent taken the measurement and the agent the measurement is taken from need to communicate with each other. Consequently, our algorithms are decentralized algorithms that do not impose restrictive network-wide connectivity condition. Moreover, we make no assumptions about the type of agents or relative measurements. We demonstrate our algorithms in simulation and a robotic~experiment. △ Less

Submitted 21 July, 2019; v1 submitted 30 April, 2019; originally announced April 2019.

Comments: 9 pages, 5 figures

Neural Processes Mixed-Effect Models for Deep Normative Modeling of Clinical Neuroimaging Data

Authors: Seyed Mostafa Kia, Andre F. Marquand

Abstract: Normative modeling has recently been introduced as a promising approach for modeling variation of neuroimaging measures across individuals in order to derive biomarkers of psychiatric disorders. Current implementations rely on Gaussian process regression, which provides coherent estimates of uncertainty needed for the method but also suffers from drawbacks including poor scaling to large datasets… ▽ More Normative modeling has recently been introduced as a promising approach for modeling variation of neuroimaging measures across individuals in order to derive biomarkers of psychiatric disorders. Current implementations rely on Gaussian process regression, which provides coherent estimates of uncertainty needed for the method but also suffers from drawbacks including poor scaling to large datasets and a reliance on fixed parametric kernels. In this paper, we propose a deep normative modeling framework based on neural processes (NPs) to solve these problems. To achieve this, we define a stochastic process formulation for mixed-effect models and show how NPs can be adopted for spatially structured mixed-effect modeling of neuroimaging data. This enables us to learn optimal feature representations and covariance structure for the random-effect and noise via global latent variables. In this scheme, predictive uncertainty can be approximated by sampling from the distribution of these global latent variables. On a publicly available clinical fMRI dataset, we compare the novelty detection performance of multivariate normative models estimated by the proposed NP approach to a baseline multi-task Gaussian process regression approach and show substantial improvements for certain diagnostic problems. △ Less

Submitted 15 April, 2019; v1 submitted 12 December, 2018; originally announced December 2018.

Comments: Medical Imaging with Deep Learning (MIDL 2019)

On the Positive Effect of Delay on the Rate of Convergence of a Class of Linear Time-Delayed Systems

Authors: Hossein Moradian, Solmaz S. Kia

Abstract: This paper is a comprehensive study of a long observed phenomenon of increase in the stability margin and so the rate of convergence of a class of linear systems due to time delay. We use Lambert W function to determine (a) in what systems the delay can lead to increase in the rate of convergence, (b) the exact range of time delay for which the rate of convergence is greater than that of the delay… ▽ More This paper is a comprehensive study of a long observed phenomenon of increase in the stability margin and so the rate of convergence of a class of linear systems due to time delay. We use Lambert W function to determine (a) in what systems the delay can lead to increase in the rate of convergence, (b) the exact range of time delay for which the rate of convergence is greater than that of the delay free system, and (c) an estimate on the value of the delay that leads to the maximum rate of convergence. For the special case when the system matrix eigenvalues are all negative real numbers, we expand our results to show that the rate of convergence in the presence of delay depends only on the eigenvalues with minimum and maximum real parts. Moreover, we determine the exact value of the maximum rate of convergence and the corresponding maximizing time delay. We demonstrate our results through a numerical example on the practical application in accelerating an agreement algorithm for networked~systems by use of a delayed feedback. △ Less

Submitted 20 July, 2019; v1 submitted 10 December, 2018; originally announced December 2018.

Scalable Multi-Task Gaussian Process Tensor Regression for Normative Modeling of Structured Variation in Neuroimaging Data

Authors: Seyed Mostafa Kia, Christian F. Beckmann, Andre F. Marquand

Abstract: Most brain disorders are very heterogeneous in terms of their underlying biology and developing analysis methods to model such heterogeneity is a major challenge. A promising approach is to use probabilistic regression methods to estimate normative models of brain function using (f)MRI data then use these to map variation across individuals in clinical populations (e.g., via anomaly detection). To… ▽ More Most brain disorders are very heterogeneous in terms of their underlying biology and developing analysis methods to model such heterogeneity is a major challenge. A promising approach is to use probabilistic regression methods to estimate normative models of brain function using (f)MRI data then use these to map variation across individuals in clinical populations (e.g., via anomaly detection). To fully capture individual differences, it is crucial to statistically model the patterns of correlation across different brain regions and individuals. However, this is very challenging for neuroimaging data because of high-dimensionality and highly structured patterns of correlation across multiple axes. Here, we propose a general and flexible multi-task learning framework to address this problem. Our model uses a tensor-variate Gaussian process in a Bayesian mixed-effects model and makes use of Kronecker algebra and a low-rank approximation to scale efficiently to multi-way neuroimaging data at the whole brain level. On a publicly available clinical fMRI dataset, we show that our computationally affordable approach substantially improves detection sensitivity over both a mass-univariate normative model and a classifier that --unlike our approach-- has full access to the clinical labels. △ Less

Submitted 30 November, 2018; v1 submitted 31 July, 2018; originally announced August 2018.

Normative Modeling of Neuroimaging Data using Scalable Multi-Task Gaussian Processes

Authors: Seyed Mostafa Kia, Andre Marquand

Abstract: Normative modeling has recently been proposed as an alternative for the case-control approach in modeling heterogeneity within clinical cohorts. Normative modeling is based on single-output Gaussian process regression that provides coherent estimates of uncertainty required by the method but does not consider spatial covariance structure. Here, we introduce a scalable multi-task Gaussian process r… ▽ More Normative modeling has recently been proposed as an alternative for the case-control approach in modeling heterogeneity within clinical cohorts. Normative modeling is based on single-output Gaussian process regression that provides coherent estimates of uncertainty required by the method but does not consider spatial covariance structure. Here, we introduce a scalable multi-task Gaussian process regression (S-MTGPR) approach to address this problem. To this end, we exploit a combination of a low-rank approximation of the spatial covariance matrix with algebraic properties of Kronecker product in order to reduce the computational complexity of Gaussian process regression in high-dimensional output spaces. On a public fMRI dataset, we show that S-MTGPR: 1) leads to substantial computational improvements that allow us to estimate normative models for high-dimensional fMRI data whilst accounting for spatial structure in data; 2) by modeling both spatial and across-sample variances, it provides higher sensitivity in novelty detection scenarios. △ Less

Submitted 5 June, 2018; v1 submitted 4 June, 2018; originally announced June 2018.

On Robustness Analysis of a Dynamic Average Consensus Algorithm to Communication Delay

Authors: Hossein Moradian, Solmaz S. Kia

Abstract: This paper studies the robustness of a dynamic average consensus algorithm to communication delay over strongly connected and weight-balanced (SCWB) digraphs. Under delay-free communication, the algorithm of interest achieves a practical asymptotic tracking of the dynamic average of the time-varying agents' reference signals. For this algorithm, in both its continuous-time and discrete-time implem… ▽ More This paper studies the robustness of a dynamic average consensus algorithm to communication delay over strongly connected and weight-balanced (SCWB) digraphs. Under delay-free communication, the algorithm of interest achieves a practical asymptotic tracking of the dynamic average of the time-varying agents' reference signals. For this algorithm, in both its continuous-time and discrete-time implementations, we characterize the admissible communication delay range and study the effect of the delay on the rate of convergence and the tracking error bound. Our study also includes establishing a relationship between the admissible delay bound and the maximum degree of the SCWB digraphs. We also show that for delays in the admissible bound, for static signals the algorithms achieve perfect tracking. Moreover, when the interaction topology is a connected undirected graph, we show that the discrete-time implementation is guaranteed to tolerate at least one step delay. Simulations demonstrate our results. △ Less

Submitted 11 December, 2018; v1 submitted 18 May, 2018; originally announced May 2018.

Deep Learning for Automatic Stereotypical Motor Movement Detection using Wearable Sensors in Autism Spectrum Disorders

Authors: Nastaran Mohammadian Rad, Seyed Mostafa Kia, Calogero Zarbo, Twan van Laarhoven, Giuseppe Jurman, Paola Venuti, Elena Marchiori, Cesare Furlanello

Abstract: Autism Spectrum Disorders are associated with atypical movements, of which stereotypical motor movements (SMMs) interfere with learning and social interaction. The automatic SMM detection using inertial measurement units (IMU) remains complex due to the strong intra and inter-subject variability, especially when handcrafted features are extracted from the signal. We propose a new application of th… ▽ More Autism Spectrum Disorders are associated with atypical movements, of which stereotypical motor movements (SMMs) interfere with learning and social interaction. The automatic SMM detection using inertial measurement units (IMU) remains complex due to the strong intra and inter-subject variability, especially when handcrafted features are extracted from the signal. We propose a new application of the deep learning to facilitate automatic SMM detection using multi-axis IMUs. We use a convolutional neural network (CNN) to learn a discriminative feature space from raw data. We show how the CNN can be used for parameter transfer learning to enhance the detection rate on longitudinal data. We also combine the long short-term memory (LSTM) with CNN to model the temporal patterns in a sequence of multi-axis signals. Further, we employ ensemble learning to combine multiple LSTM learners into a more robust SMM detector. Our results show that: 1) feature learning outperforms handcrafted features; 2) parameter transfer learning is beneficial in longitudinal settings; 3) using LSTM to learn the temporal dynamic of signals enhances the detection rate especially for skewed training data; 4) an ensemble of LSTMs provides more accurate and stable detectors. These findings provide a significant step toward accurate SMM detection in real-time scenarios. △ Less

Submitted 14 September, 2017; originally announced September 2017.

Gender and Emotion Recognition with Implicit User Signals

Authors: Maneesh Bilalpur, Seyed Mostafa Kia, Manisha Chawla, Tat-Seng Chua, Ramanathan Subramanian

Abstract: We examine the utility of implicit user behavioral signals captured using low-cost, off-the-shelf devices for anonymous gender and emotion recognition. A user study designed to examine male and female sensitivity to facial emotions confirms that females recognize (especially negative) emotions quicker and more accurately than men, mirroring prior findings. Implicit viewer responses in the form of… ▽ More We examine the utility of implicit user behavioral signals captured using low-cost, off-the-shelf devices for anonymous gender and emotion recognition. A user study designed to examine male and female sensitivity to facial emotions confirms that females recognize (especially negative) emotions quicker and more accurately than men, mirroring prior findings. Implicit viewer responses in the form of EEG brain signals and eye movements are then examined for existence of (a) emotion and gender-specific patterns from event-related potentials (ERPs) and fixation distributions and (b) emotion and gender discriminability. Experiments reveal that (i) Gender and emotion-specific differences are observable from ERPs, (ii) multiple similarities exist between explicit responses gathered from users and their implicit behavioral signals, and (iii) Significantly above-chance ($\approx$70%) gender recognition is achievable on comparing emotion-specific EEG responses-- gender differences are encoded best for anger and disgust. Also, fairly modest valence (positive vs negative emotion) recognition is achieved with EEG and eye-based features. △ Less

Submitted 29 August, 2017; originally announced August 2017.

Comments: To be published in the Proceedings of 19th International Conference on Multimodal Interaction.2017

ACM Class: H.5.2; I.3.6; H.1.2

Discovering Gender Differences in Facial Emotion Recognition via Implicit Behavioral Cues

Authors: Maneesh Bilalpur, Seyed Mostafa Kia, Tat-Seng Chua, Ramanathan Subramanian

Abstract: We examine the utility of implicit behavioral cues in the form of EEG brain signals and eye movements for gender recognition (GR) and emotion recognition (ER). Specifically, the examined cues are acquired via low-cost, off-the-shelf sensors. We asked 28 viewers (14 female) to recognize emotions from unoccluded (no mask) as well as partially occluded (eye and mouth masked) emotive faces. Obtained e… ▽ More We examine the utility of implicit behavioral cues in the form of EEG brain signals and eye movements for gender recognition (GR) and emotion recognition (ER). Specifically, the examined cues are acquired via low-cost, off-the-shelf sensors. We asked 28 viewers (14 female) to recognize emotions from unoccluded (no mask) as well as partially occluded (eye and mouth masked) emotive faces. Obtained experimental results reveal that (a) reliable GR and ER is achievable with EEG and eye features, (b) differential cognitive processing especially for negative emotions is observed for males and females and (c) some of these cognitive differences manifest under partial face occlusion, as typified by the eye and mouth mask conditions. △ Less

Submitted 29 August, 2017; originally announced August 2017.

Comments: To be published in the Proceedings of Seventh International Conference on Affective Computing and Intelligent Interaction.2017

ACM Class: H.5.2; I.3.6; H.1.2

Server assisted distributed cooperative localization over unreliable communication links

Authors: Solmaz S. Kia, Jonathan Hechtbauer, David Gogokhiya, Sonia Martinez

Abstract: This paper considers the problem of cooperative localization (CL) using inter-robot measurements for a group of networked robots with limited on-board resources. We propose a novel recursive algorithm in which each robot localizes itself in a global coordinate frame by local dead reckoning, and opportunistically corrects its pose estimate whenever it receives a relative measurement update message… ▽ More This paper considers the problem of cooperative localization (CL) using inter-robot measurements for a group of networked robots with limited on-board resources. We propose a novel recursive algorithm in which each robot localizes itself in a global coordinate frame by local dead reckoning, and opportunistically corrects its pose estimate whenever it receives a relative measurement update message from a server. The computation and storage cost per robot in terms of the size of the team is of order O(1), and the robots are only required to transmit information when they are involved in a relative measurement. The server also only needs to compute and transmit update messages when it receives an inter-robot measurement. We show that under perfect communication, our algorithm is an alternative but exact implementation of a joint CL for the entire team via Extended Kalman Filter (EKF). The perfect communication however is not a hard requirement. In fact, we show that our algorithm is intrinsically robust with respect to communication failures, with formal guarantees that the updated estimates of the robots receiving the update message are of minimum variance in a first-order approximate sense at that given timestep. We demonstrate the performance of the algorithm in simulation and experiments. △ Less

Submitted 24 December, 2017; v1 submitted 1 August, 2016; originally announced August 2016.

Comments: The title has changes from "A partially decentralized EKF scheme for cooperative localization over unreliable communication links" to "Server assisted distributed cooperative localization over unreliable communication links". The presentation of the paper is revised. New example is added. Experimental results are added

Interpretability of Multivariate Brain Maps in Brain Decoding: Definition and Quantification

Authors: Seyed Mostafa Kia

Abstract: Brain decoding is a popular multivariate approach for hypothesis testing in neuroimaging. It is well known that the brain maps derived from weights of linear classifiers are hard to interpret because of high correlations between predictors, low signal to noise ratios, and the high dimensionality of neuroimaging data. Therefore, improving the interpretability of brain decoding approaches is of prim… ▽ More Brain decoding is a popular multivariate approach for hypothesis testing in neuroimaging. It is well known that the brain maps derived from weights of linear classifiers are hard to interpret because of high correlations between predictors, low signal to noise ratios, and the high dimensionality of neuroimaging data. Therefore, improving the interpretability of brain decoding approaches is of primary interest in many neuroimaging studies. Despite extensive studies of this type, at present, there is no formal definition for interpretability of multivariate brain maps. As a consequence, there is no quantitative measure for evaluating the interpretability of different brain decoding methods. In this paper, first, we present a theoretical definition of interpretability in brain decoding; we show that the interpretability of multivariate brain maps can be decomposed into their reproducibility and representativeness. Second, as an application of the proposed theoretical definition, we formalize a heuristic method for approximating the interpretability of multivariate brain maps in a binary magnetoencephalography (MEG) decoding scenario. Third, we propose to combine the approximated interpretability and the performance of the brain decoding model into a new multi-objective criterion for model selection. Our results for the MEG data show that optimizing the hyper-parameters of the regularized linear classifier based on the proposed criterion results in more informative multivariate brain maps. More importantly, the presented definition provides the theoretical background for quantitative evaluation of interpretability, and hence, facilitates the development of more effective brain decoding algorithms in the future. △ Less

Submitted 29 March, 2016; originally announced March 2016.

Convolutional Neural Network for Stereotypical Motor Movement Detection in Autism

Authors: Nastaran Mohammadian Rad, Andrea Bizzego, Seyed Mostafa Kia, Giuseppe Jurman, Paola Venuti, Cesare Furlanello

Abstract: Autism Spectrum Disorders (ASDs) are often associated with specific atypical postural or motor behaviors, of which Stereotypical Motor Movements (SMMs) have a specific visibility. While the identification and the quantification of SMM patterns remain complex, its automation would provide support to accurate tuning of the intervention in the therapy of autism. Therefore, it is essential to develop… ▽ More Autism Spectrum Disorders (ASDs) are often associated with specific atypical postural or motor behaviors, of which Stereotypical Motor Movements (SMMs) have a specific visibility. While the identification and the quantification of SMM patterns remain complex, its automation would provide support to accurate tuning of the intervention in the therapy of autism. Therefore, it is essential to develop automatic SMM detection systems in a real world setting, taking care of strong inter-subject and intra-subject variability. Wireless accelerometer sensing technology can provide a valid infrastructure for real-time SMM detection, however such variability remains a problem also for machine learning methods, in particular whenever handcrafted features extracted from accelerometer signal are considered. Here, we propose to employ the deep learning paradigm in order to learn discriminating features from multi-sensor accelerometer signals. Our results provide preliminary evidence that feature learning and transfer learning embedded in the deep architecture achieve higher accurate SMM detectors in longitudinal scenarios. △ Less

Submitted 7 June, 2016; v1 submitted 5 November, 2015; originally announced November 2015.

Comments: Presented at 5th NIPS Workshop on Machine Learning and Interpretation in Neuroimaging (MLINI), 2015, (https://arxiv.boxedpaper.com/html/1605.04435), Report-no: MLINI/2015/13

Report number: MLINI/2015/13

Cooperative localization for mobile agents: a recursive decentralized algorithm based on Kalman filter decoupling

Authors: Solmaz S. Kia, Stephen Rounds, Sonia Martinez

Abstract: We consider cooperative localization technique for mobile agents with communication and computation capabilities. We start by provide and overview of different decentralization strategies in the literature, with special focus on how these algorithms maintain an account of intrinsic correlations between state estimate of team members. Then, we present a novel decentralized cooperative localization… ▽ More We consider cooperative localization technique for mobile agents with communication and computation capabilities. We start by provide and overview of different decentralization strategies in the literature, with special focus on how these algorithms maintain an account of intrinsic correlations between state estimate of team members. Then, we present a novel decentralized cooperative localization algorithm that is a decentralized implementation of a centralized Extended Kalman Filter for cooperative localization. In this algorithm, instead of propagating cross-covariance terms, each agent propagates new intermediate local variables that can be used in an update stage to create the required propagated cross-covariance terms. Whenever there is a relative measurement in the network, the algorithm declares the agent making this measurement as the interim master. By acquiring information from the interim landmark, the agent the relative measurement is taken from, the interim master can calculate and broadcast a set of intermediate variables which each robot can then use to update its estimates to match that of a centralized Extended Kalman Filter for cooperative localization. Once an update is done, no further communication is needed until the next relative measurement. △ Less

Submitted 5 October, 2015; v1 submitted 21 May, 2015; originally announced May 2015.

Mass-Univariate Hypothesis Testing on MEEG Data using Cross-Validation

Authors: Seyed Mostafa Kia

Abstract: Recent advances in statistical theory, together with advances in the computational power of computers, provide alternative methods to do mass-univariate hypothesis testing in which a large number of univariate tests, can be properly used to compare MEEG data at a large number of time-frequency points and scalp locations. One of the major problematic aspects of this kind of mass-univariate analysis… ▽ More Recent advances in statistical theory, together with advances in the computational power of computers, provide alternative methods to do mass-univariate hypothesis testing in which a large number of univariate tests, can be properly used to compare MEEG data at a large number of time-frequency points and scalp locations. One of the major problematic aspects of this kind of mass-univariate analysis is due to high number of accomplished hypothesis tests. Hence procedures that remove or alleviate the increased probability of false discoveries are crucial for this type of analysis. Here, I propose a new method for mass-univariate analysis of MEEG data based on cross-validation scheme. In this method, I suggest a hierarchical classification procedure under k-fold cross-validation to detect which sensors at which time-bin and which frequency-bin contributes in discriminating between two different stimuli or tasks. To achieve this goal, a new feature extraction method based on the discrete cosine transform (DCT) employed to get maximum advantage of all three data dimensions. Employing cross-validation and hierarchy architecture alongside the DCT feature space makes this method more reliable and at the same time enough sensitive to detect the narrow effects in brain activities. △ Less

Submitted 25 June, 2014; originally announced June 2014.

Comments: Master thesis, July 2013

MEG Decoding Across Subjects

Authors: Emanuele Olivetti, Seyed Mostafa Kia, Paolo Avesani

Abstract: Brain decoding is a data analysis paradigm for neuroimaging experiments that is based on predicting the stimulus presented to the subject from the concurrent brain activity. In order to make inference at the group level, a straightforward but sometimes unsuccessful approach is to train a classifier on the trials of a group of subjects and then to test it on unseen trials from new subjects. The ext… ▽ More Brain decoding is a data analysis paradigm for neuroimaging experiments that is based on predicting the stimulus presented to the subject from the concurrent brain activity. In order to make inference at the group level, a straightforward but sometimes unsuccessful approach is to train a classifier on the trials of a group of subjects and then to test it on unseen trials from new subjects. The extreme difficulty is related to the structural and functional variability across the subjects. We call this approach "decoding across subjects". In this work, we address the problem of decoding across subjects for magnetoencephalographic (MEG) experiments and we provide the following contributions: first, we formally describe the problem and show that it belongs to a machine learning sub-field called transductive transfer learning (TTL). Second, we propose to use a simple TTL technique that accounts for the differences between train data and test data. Third, we propose the use of ensemble learning, and specifically of stacked generalization, to address the variability across subjects within train data, with the aim of producing more stable classifiers. On a face vs. scramble task MEG dataset of 16 subjects, we compare the standard approach of not modelling the differences across subjects, to the proposed one of combining TTL and ensemble learning. We show that the proposed approach is consistently more accurate than the standard one. △ Less

Submitted 16 April, 2014; originally announced April 2014.

A Novel Scheme for Intelligent Recognition of Pornographic Images

Authors: Seyed Mostafa Kia, Hossein Rahmani, Reza Mortezaei, Mohsen Ebrahimi Moghaddam, Amer Namazi

Abstract: Harmful contents are rising in internet day by day and this motivates the essence of more research in fast and reliable obscene and immoral material filtering. Pornographic image recognition is an important component in each filtering system. In this paper, a new approach for detecting pornographic images is introduced. In this approach, two new features are suggested. These two features in combin… ▽ More Harmful contents are rising in internet day by day and this motivates the essence of more research in fast and reliable obscene and immoral material filtering. Pornographic image recognition is an important component in each filtering system. In this paper, a new approach for detecting pornographic images is introduced. In this approach, two new features are suggested. These two features in combination with other simple traditional features provide decent difference between porn and non-porn images. In addition, we applied fuzzy integral based information fusion to combine MLP (Multi-Layer Perceptron) and NF (Neuro-Fuzzy) outputs. To test the proposed method, performance of system was evaluated over 18354 download images from internet. The attained precision was 93% in TP and 8% in FP on training dataset, and 87% and 5.5% on test dataset. Achieved results verify the performance of proposed system versus other related works. △ Less

Submitted 29 September, 2014; v1 submitted 24 February, 2014; originally announced February 2014.