-
A Hybrid Systems Model of Feedback Optimization for Linear Systems
Authors:
Oscar Jed Chuy,
Matthew Hale,
Ricardo Sanfelice
Abstract:
Feedback optimization algorithms compute inputs to a system in real time, which helps mitigate the effects of unknown disturbances. However, existing work models both system dynamics and computations in either discrete or continuous time, which does not faithfully model some applications. In this work, we model linear system dynamics in continuous time, and we model the computations of inputs in d…
▽ More
Feedback optimization algorithms compute inputs to a system in real time, which helps mitigate the effects of unknown disturbances. However, existing work models both system dynamics and computations in either discrete or continuous time, which does not faithfully model some applications. In this work, we model linear system dynamics in continuous time, and we model the computations of inputs in discrete time. Therefore, we present a novel hybrid systems framework for modeling feedback optimization of linear time-invariant systems that are subject to unknown, constant disturbances. For this setup, we first establish the well-posedness of the hybrid model and establish completeness of solutions while ruling out Zeno behavior. Then, our main result derives a convergence rate and an error bound for the full hybrid computation-in-theloop system and shows that it converges exponentially towards a ball of known radius about a desired fixed point. Simulation results show that this approach successfully mitigates the effects of disturbances, with the magnitude of steady-state error being 81% less than the magnitude of the disturbances in the system.
△ Less
Submitted 22 May, 2025; v1 submitted 31 March, 2025;
originally announced April 2025.
-
Verification and Synthesis Methods for High-Order Control Barrier Functions
Authors:
Ellie Pond,
Matthew Hale
Abstract:
High-order control barrier functions (HOCBFs) can be used to provide autonomous systems with safety, though computational methods to verify and synthesize these functions remain lacking. In this work, we address this need by formulating SOS programs that verify and synthesize HOCBFs, such that continued safety is always guaranteed forward in time. We first propose a verification SOS program for sy…
▽ More
High-order control barrier functions (HOCBFs) can be used to provide autonomous systems with safety, though computational methods to verify and synthesize these functions remain lacking. In this work, we address this need by formulating SOS programs that verify and synthesize HOCBFs, such that continued safety is always guaranteed forward in time. We first propose a verification SOS program for systems with (i) one or multiple HOCBFs, (ii) a control Lyapunov function (CLF), and (iii) input constraints, and we show that a solution to this problem guarantees that the online implementation of the system is always safe. Next, we propose a sequence of SOS programs that synthesize the class K functions used in an HOCBF, and we show that this sequence of problems ensures that a system is guaranteed to remain safe while running. After that, a synthesis framework is given that ensures real-time safety for systems with (i) multiple HOCBFs, (ii) a CLF, and (iii) input constraints. Our developments are illustrated in numerical simulations for a system with seven HOCBFs of maximum relative degree two, with 14 total unknown class K functions, all of which are successfully synthesized in a way that produces safe autonomy.
△ Less
Submitted 5 February, 2025;
originally announced February 2025.
-
Scalable Distributed Reproduction Numbers of Network Epidemics with Differential Privacy
Authors:
Bo Chen,
Baike She,
Calvin Hawkins,
Philip E. Paré,
Matthew T. Hale
Abstract:
Reproduction numbers are widely used for the estimation and prediction of epidemic spreading processes over networks. However, conventional reproduction numbers of an overall network do not indicate where an epidemic is spreading. Therefore, we propose a novel notion of local distributed reproduction numbers to capture the spreading behaviors of each node in a network. We first show how to compute…
▽ More
Reproduction numbers are widely used for the estimation and prediction of epidemic spreading processes over networks. However, conventional reproduction numbers of an overall network do not indicate where an epidemic is spreading. Therefore, we propose a novel notion of local distributed reproduction numbers to capture the spreading behaviors of each node in a network. We first show how to compute them and then use them to derive new conditions under which an outbreak can occur. These conditions are then used to derive new conditions for the existence, uniqueness, and stability of equilibrium states of the underlying epidemic model. Building upon these local distributed reproduction numbers, we define cluster distributed reproduction numbers to model the spread between clusters composed of nodes. Furthermore, we demonstrate that the local distributed reproduction numbers can be aggregated into cluster distributed reproduction numbers at different scales. However, both local and cluster distributed reproduction numbers can reveal the frequency of interactions between nodes in a network, which raises privacy concerns. Thus, we next develop a privacy framework that implements a differential privacy mechanism to provably protect the frequency of interactions between nodes when computing distributed reproduction numbers. Numerical experiments show that, even under differential privacy, the distributed reproduction numbers provide accurate estimates of the epidemic spread while also providing more insights than conventional reproduction numbers.
△ Less
Submitted 3 February, 2025; v1 submitted 30 January, 2025;
originally announced January 2025.
-
Time-Constrained Model Predictive Control for Autonomous Satellite Rendezvous, Proximity Operations, and Docking
Authors:
Gabriel Behrendt,
Matthew Hale,
Alexander Soderlund,
Sean Phillips,
Evan Kain
Abstract:
This paper presents a time-constrained model predictive control strategy for the six degree-of-freedom autonomous rendezvous, proximity, operations and docking problem between a controllable "deputy" satellite and an uncontrolled "chief" satellite. The objective is to achieve a docking configuration defined by both the translational and attitudinal states of the deputy relative to the chief, whose…
▽ More
This paper presents a time-constrained model predictive control strategy for the six degree-of-freedom autonomous rendezvous, proximity, operations and docking problem between a controllable "deputy" satellite and an uncontrolled "chief" satellite. The objective is to achieve a docking configuration defined by both the translational and attitudinal states of the deputy relative to the chief, whose dynamics are respectively governed by both the Clohessy-Wiltshire equations and Euler's second law of motion. The proposed control strategy explicitly addresses computational time constraints that are common to state-of-the-art space vehicles. Thus, a time-constrained model predictive control strategy is implemented on a space-grade processor. Although suboptimal with regards to energy consumption when compared to conventional optimal RPO trajectories, it is empirically demonstrated via numerical simulations that the deputy spacecraft still achieves a successful docking configuration while subject to computational time constraints.
△ Less
Submitted 22 January, 2025;
originally announced January 2025.
-
A Dissipativity Approach to Analyzing Composite Spreading Networks
Authors:
Baike She,
Matthew Hale
Abstract:
The study of spreading processes often analyzes networks at different resolutions, e.g., at the level of individuals or countries, but it is not always clear how properties at one resolution can carry over to another. Accordingly, in this work we use dissipativity theory from control system analysis to characterize composite spreading networks that are comprised by many interacting subnetworks. We…
▽ More
The study of spreading processes often analyzes networks at different resolutions, e.g., at the level of individuals or countries, but it is not always clear how properties at one resolution can carry over to another. Accordingly, in this work we use dissipativity theory from control system analysis to characterize composite spreading networks that are comprised by many interacting subnetworks. We first develop a method to represent spreading networks that have inputs and outputs. Then we define a composition operation for composing multiple spreading networks into a larger composite spreading network. Next, we develop storage and supply rate functions that can be used to demonstrate that spreading dynamics are dissipative. We then derive conditions under which a composite spreading network will converge to a disease-free equilibrium as long as its constituent spreading networks are dissipative with respect to those storage and supply rate functions. To illustrate these results, we use simulations of an influenza outbreak in a primary school, and we show that an outbreak can be prevented by decreasing the average interaction time between any pair of classes to less than 79% of the original interaction time.
△ Less
Submitted 3 December, 2024;
originally announced December 2024.
-
Technical Report: A Totally Asynchronous Nesterov's Accelerated Gradient Method for Convex Optimization
Authors:
Ellie Pond,
April Sebok,
Zachary Bell,
Matthew Hale
Abstract:
We present a totally asynchronous algorithm for convex optimization that is based on a novel generalization of Nesterov's accelerated gradient method. This algorithm is developed for fast convergence under "total asynchrony," i.e., allowing arbitrarily long delays between agents' computations and communications without assuming any form of delay bound. These conditions may arise, for example, due…
▽ More
We present a totally asynchronous algorithm for convex optimization that is based on a novel generalization of Nesterov's accelerated gradient method. This algorithm is developed for fast convergence under "total asynchrony," i.e., allowing arbitrarily long delays between agents' computations and communications without assuming any form of delay bound. These conditions may arise, for example, due to jamming by adversaries. Our framework is block-based, in the sense that each agent is only responsible for computing updates to (and communicating the values of) a small subset of the network-level decision variables. In our main result, we present bounds on the algorithm's parameters that guarantee linear convergence to an optimizer. Then, we quantify the relationship between (i) the total number of computations and communications executed by the agents and (ii) the agents' collective distance to an optimum. Numerical simulations show that this algorithm requires 28% fewer iterations than the heavy ball algorithm and 61% fewer iterations than gradient descent under total asynchrony.
△ Less
Submitted 14 June, 2024;
originally announced June 2024.
-
Modeling Epidemic Spread: A Gaussian Process Regression Approach
Authors:
Baike She,
Lei Xin,
Philip E. Paré,
Matthew Hale
Abstract:
Modeling epidemic spread is critical for informing policy decisions aimed at mitigation. Accordingly, in this work we present a new data-driven method based on Gaussian process regression (GPR) to model epidemic spread through the difference on the logarithmic scale of the infected cases. We bound the variance of the predictions made by GPR, which quantifies the impact of epidemic data on the prop…
▽ More
Modeling epidemic spread is critical for informing policy decisions aimed at mitigation. Accordingly, in this work we present a new data-driven method based on Gaussian process regression (GPR) to model epidemic spread through the difference on the logarithmic scale of the infected cases. We bound the variance of the predictions made by GPR, which quantifies the impact of epidemic data on the proposed model. Next, we derive a high-probability error bound on the prediction error in terms of the distance between the training points and a testing point, the posterior variance, and the level of change in the spreading process, and we assess how the characteristics of the epidemic spread and infection data influence this error bound. We present examples that use GPR to model and predict epidemic spread by using real-world infection data gathered in the UK during the COVID-19 epidemic. These examples illustrate that, under typical conditions, the prediction for the next twenty days has 94.29% of the noisy data located within the 95% confidence interval, validating these predictions. We further compare the modeling and prediction results with other methods, such as polynomial regression, k-nearest neighbors (KNN) regression, and neural networks, to demonstrate the benefits of leveraging GPR in disease spread modeling.
△ Less
Submitted 21 November, 2024; v1 submitted 14 December, 2023;
originally announced December 2023.
-
Differentially Private Reward Functions in Policy Synthesis for Markov Decision Processes
Authors:
Alexander Benvenuti,
Calvin Hawkins,
Brandon Fallin,
Bo Chen,
Brendan Bialy,
Miriam Dennis,
Matthew Hale
Abstract:
Markov decision processes often seek to maximize a reward function, but onlookers may infer reward functions by observing the states and actions of such systems, revealing sensitive information. Therefore, in this paper we introduce and compare two methods for privatizing reward functions in policy synthesis for multi-agent Markov decision processes, which generalize Markov decision processes. Rew…
▽ More
Markov decision processes often seek to maximize a reward function, but onlookers may infer reward functions by observing the states and actions of such systems, revealing sensitive information. Therefore, in this paper we introduce and compare two methods for privatizing reward functions in policy synthesis for multi-agent Markov decision processes, which generalize Markov decision processes. Reward functions are privatized using differential privacy, a statistical framework for protecting sensitive data. The methods we develop perturb either (1) each agent's individual reward function or (2) the joint reward function shared by all agents. We show that approach (1) provides better performance. We then develop a polynomial-time algorithm for the numerical computation of the performance loss due to privacy on a case-by-case basis. Next, using approach (1), we develop guidelines for selecting reward function values to preserve ``goal" and ``avoid" states while still remaining private, and we quantify the increase in computational complexity needed to compute policies from privatized rewards. Numerical simulations are performed on three classes of systems and they reveal a surprising compatibility with privacy: using reasonably strong privacy ($ε=1.3$) on average induces as little as a~$5\%$ decrease in total accumulated reward and a $0.016\%$ increase in computation time.
△ Less
Submitted 3 September, 2024; v1 submitted 21 September, 2023;
originally announced September 2023.
-
DOMINO++: Domain-aware Loss Regularization for Deep Learning Generalizability
Authors:
Skylar E. Stolte,
Kyle Volle,
Aprinda Indahlastari,
Alejandro Albizu,
Adam J. Woods,
Kevin Brink,
Matthew Hale,
Ruogu Fang
Abstract:
Out-of-distribution (OOD) generalization poses a serious challenge for modern deep learning (DL). OOD data consists of test data that is significantly different from the model's training data. DL models that perform well on in-domain test data could struggle on OOD data. Overcoming this discrepancy is essential to the reliable deployment of DL. Proper model calibration decreases the number of spur…
▽ More
Out-of-distribution (OOD) generalization poses a serious challenge for modern deep learning (DL). OOD data consists of test data that is significantly different from the model's training data. DL models that perform well on in-domain test data could struggle on OOD data. Overcoming this discrepancy is essential to the reliable deployment of DL. Proper model calibration decreases the number of spurious connections that are made between model features and class outputs. Hence, calibrated DL can improve OOD generalization by only learning features that are truly indicative of the respective classes. Previous work proposed domain-aware model calibration (DOMINO) to improve DL calibration, but it lacks designs for model generalizability to OOD data. In this work, we propose DOMINO++, a dual-guidance and dynamic domain-aware loss regularization focused on OOD generalizability. DOMINO++ integrates expert-guided and data-guided knowledge in its regularization. Unlike DOMINO which imposed a fixed scaling and regularization rate, DOMINO++ designs a dynamic scaling factor and an adaptive regularization rate. Comprehensive evaluations compare DOMINO++ with DOMINO and the baseline model for head tissue segmentation from magnetic resonance images (MRIs) on OOD data. The OOD data consists of synthetic noisy and rotated datasets, as well as real data using a different MRI scanner from a separate site. DOMINO++'s superior performance demonstrates its potential to improve the trustworthy deployment of DL on real clinical data.
△ Less
Submitted 20 August, 2023;
originally announced August 2023.
-
Characterizing Compositionality of LQR from the Categorical Perspective
Authors:
Baike She,
Tyler Hanks,
James Fairbanks,
Matthew Hale
Abstract:
Composing systems is a fundamental concept in modern control systems, yet it remains challenging to formally analyze how controllers designed for individual subsystems can differ from controllers designed for the composition of those subsystems. To address this challenge, we propose a novel approach to composing control systems based on resource sharing machines, a concept from applied category th…
▽ More
Composing systems is a fundamental concept in modern control systems, yet it remains challenging to formally analyze how controllers designed for individual subsystems can differ from controllers designed for the composition of those subsystems. To address this challenge, we propose a novel approach to composing control systems based on resource sharing machines, a concept from applied category theory. We use resource sharing machines to investigate the differences between (i) the linear-quadratic regulator (LQR) designed directly for a composite system and (ii) the LQR that is attained through the composition of LQRs designed for each subsystem. We first establish novel formalisms to compose LQR control designs using resource sharing machines. Then we develop new sufficient conditions to guarantee that the LQR designed for a composite system is equal to the LQR attained through composition of LQRs for its subsystems. In addition, we reduce the developed condition to that of checking the controllability and observability of a certain linear, time-invariant system, which provides a simple, computationally efficient procedure for evaluating the equivalence of controllers for composed systems.
△ Less
Submitted 2 May, 2023;
originally announced May 2023.
-
Anomaly Search Over Many Sequences With Switching Costs
Authors:
Matthew Ubl,
Benjamin D. Robinson,
Matthew T. Hale
Abstract:
This paper considers the quickest search problem to identify anomalies among large numbers of data streams. These streams can model, for example, disjoint regions monitored by a mobile robot. A particular challenge is a version of the problem in which the experimenter must suffer a cost each time the data stream being sampled changes, such as the time the robot must spend moving between regions. I…
▽ More
This paper considers the quickest search problem to identify anomalies among large numbers of data streams. These streams can model, for example, disjoint regions monitored by a mobile robot. A particular challenge is a version of the problem in which the experimenter must suffer a cost each time the data stream being sampled changes, such as the time the robot must spend moving between regions. In this paper, we propose an algorithm which accounts for switching costs by varying a confidence threshold that governs when the algorithm switches to a new data stream. Our main contributions are easily computable approximations for both the optimal value of this threshold and the optimal value of the parameter that determines when a stream must be re-sampled. Further, we empirically show (i) a uniform improvement for switching costs of interest and (ii) roughly equivalent performance for small switching costs when comparing to the closest available algorithm.
△ Less
Submitted 16 March, 2023;
originally announced March 2023.
-
Distributed Reproduction Numbers of Networked Epidemics
Authors:
Baike She,
Philip E. Paré,
Matthew Hale
Abstract:
Reproduction numbers are widely used for the estimation and prediction of epidemic spreading processes over networks. However, reproduction numbers do not enable estimation and prediction in individual communities within networks, and they can be difficult to compute due to the aggregation of infection data that is required to do so. Therefore, in this work we propose a novel concept of distribute…
▽ More
Reproduction numbers are widely used for the estimation and prediction of epidemic spreading processes over networks. However, reproduction numbers do not enable estimation and prediction in individual communities within networks, and they can be difficult to compute due to the aggregation of infection data that is required to do so. Therefore, in this work we propose a novel concept of distributed reproduction numbers to capture the spreading behaviors of each entity in the network, and we show how to compute them using certain parameters in networked SIS and SIR epidemic models. We use distributed reproduction numbers to derive new conditions under which an outbreak can occur. These conditions are then used to derive new conditions for the existence, uniqueness, and stability of equilibrium states. Finally, in simulation we use synthetic infection data to illustrate how distributed reproduction numbers provide more fine-grained analyses of networked spreading processes than ordinary reproduction numbers.
△ Less
Submitted 18 January, 2023;
originally announced January 2023.
-
Fast Verification of Control Barrier Functions via Linear Programming
Authors:
Ellie Pond,
Matthew Hale
Abstract:
Control barrier functions are a popular method of ensuring system safety, and these functions can be used to enforce invariance of a set under the dynamics of a system. A control barrier function must have certain properties, and one must both formulate a candidate control barrier function and verify that it does indeed satisfy the required properties. Targeting the latter problem, this paper pres…
▽ More
Control barrier functions are a popular method of ensuring system safety, and these functions can be used to enforce invariance of a set under the dynamics of a system. A control barrier function must have certain properties, and one must both formulate a candidate control barrier function and verify that it does indeed satisfy the required properties. Targeting the latter problem, this paper presents a method of verifying any finite number of candidate control barrier functions with linear programming. We first apply techniques from real algebraic geometry to formulate verification problem statements that are solvable numerically. Typically, semidefinite programming is used to verify candidate control barrier functions, but this does not always scale well. Therefore, we next apply a method of inner-approximating the set of sums of squares polynomials that significantly reduces the computational complexity of these verification problems by transcribing them to linear programs. We give explicit forms for the resulting linear programs, and simulation results for a satellite inspection problem show that the computation time needed for verification can be reduced by more than 95%.
△ Less
Submitted 1 December, 2022;
originally announced December 2022.
-
DOMINO: Domain-aware Model Calibration in Medical Image Segmentation
Authors:
Skylar E. Stolte,
Kyle Volle,
Aprinda Indahlastari,
Alejandro Albizu,
Adam J. Woods,
Kevin Brink,
Matthew Hale,
Ruogu Fang
Abstract:
Model calibration measures the agreement between the predicted probability estimates and the true correctness likelihood. Proper model calibration is vital for high-risk applications. Unfortunately, modern deep neural networks are poorly calibrated, compromising trustworthiness and reliability. Medical image segmentation particularly suffers from this due to the natural uncertainty of tissue bound…
▽ More
Model calibration measures the agreement between the predicted probability estimates and the true correctness likelihood. Proper model calibration is vital for high-risk applications. Unfortunately, modern deep neural networks are poorly calibrated, compromising trustworthiness and reliability. Medical image segmentation particularly suffers from this due to the natural uncertainty of tissue boundaries. This is exasperated by their loss functions, which favor overconfidence in the majority classes. We address these challenges with DOMINO, a domain-aware model calibration method that leverages the semantic confusability and hierarchical similarity between class labels. Our experiments demonstrate that our DOMINO-calibrated deep neural networks outperform non-calibrated models and state-of-the-art morphometric methods in head image segmentation. Our results show that our method can consistently achieve better calibration, higher accuracy, and faster inference times than these methods, especially on rarer classes. This performance is attributed to our domain-aware regularization to inform semantic model calibration. These findings show the importance of semantic ties between class labels in building confidence in deep learning models. The framework has the potential to improve the trustworthiness and reliability of generic medical image segmentation models. The code for this article is available at: https://github.com/lab-smile/DOMINO.
△ Less
Submitted 13 September, 2022;
originally announced September 2022.
-
Differential Privacy for Symbolic Systems with Application to Markov Chains
Authors:
Bo Chen,
Kevin Leahy,
Austin Jones,
Matthew Hale
Abstract:
Data-driven systems are gathering increasing amounts of data from users, and sensitive user data requires privacy protections. In some cases, the data gathered is non-numerical or symbolic, and conventional approaches to privacy, e.g., adding noise, do not apply, though such systems still require privacy protections. Accordingly, we present a novel differential privacy framework for protecting tra…
▽ More
Data-driven systems are gathering increasing amounts of data from users, and sensitive user data requires privacy protections. In some cases, the data gathered is non-numerical or symbolic, and conventional approaches to privacy, e.g., adding noise, do not apply, though such systems still require privacy protections. Accordingly, we present a novel differential privacy framework for protecting trajectories generated by symbolic systems. These trajectories can be represented as words or strings over a finite alphabet. We develop new differential privacy mechanisms that approximate a sensitive word using a random word that is likely to be near it. An offline mechanism is implemented efficiently using a Modified Hamming Distance Automaton to generate whole privatized output words over a finite time horizon. Then, an online mechanism is implemented by taking in a sensitive symbol and generating a randomized output symbol at each timestep. This work is extended to Markov chains to generate differentially private state sequences that a given Markov chain could have produced. Statistical accuracy bounds are developed to quantify the accuracy of these mechanisms, and numerical results validate the accuracy of these techniques for strings of English words.
△ Less
Submitted 11 August, 2022; v1 submitted 7 February, 2022;
originally announced February 2022.
-
Edge Differential Privacy for Algebraic Connectivity of Graphs
Authors:
Bo Chen,
Calvin Hawkins,
Kasra Yazdani,
Matthew Hale
Abstract:
Graphs are the dominant formalism for modeling multi-agent systems. The algebraic connectivity of a graph is particularly important because it provides the convergence rates of consensus algorithms that underlie many multi-agent control and optimization techniques. However, sharing the value of algebraic connectivity can inadvertently reveal sensitive information about the topology of a graph, suc…
▽ More
Graphs are the dominant formalism for modeling multi-agent systems. The algebraic connectivity of a graph is particularly important because it provides the convergence rates of consensus algorithms that underlie many multi-agent control and optimization techniques. However, sharing the value of algebraic connectivity can inadvertently reveal sensitive information about the topology of a graph, such as connections in social networks. Therefore, in this work we present a method to release a graph's algebraic connectivity under a graph-theoretic form of differential privacy, called edge differential privacy. Edge differential privacy obfuscates differences among graphs' edge sets and thus conceals the absence or presence of sensitive connections therein. We provide privacy with bounded Laplace noise, which improves accuracy relative to conventional unbounded noise. The private algebraic connectivity values are analytically shown to provide accurate estimates of consensus convergence rates, as well as accurate bounds on the diameter of a graph and the mean distance between its nodes. Simulation results confirm the utility of private algebraic connectivity in these contexts.
△ Less
Submitted 1 April, 2021;
originally announced April 2021.
-
Predictive resource allocation for flexible loads with local QoS
Authors:
Austin R. Coffman,
Matthew Hale,
Prabir Barooah
Abstract:
Loads that can vary their power consumption without violating their Quality of service (QoS), that is flexible loads, are an invaluable resource for grid operators. Utilizing flexible loads as a resource requires the grid operator to incorporate them into a resource allocation problem. Since flexible loads are often consumers, for concerns of privacy it is desirable for this problem to have a dist…
▽ More
Loads that can vary their power consumption without violating their Quality of service (QoS), that is flexible loads, are an invaluable resource for grid operators. Utilizing flexible loads as a resource requires the grid operator to incorporate them into a resource allocation problem. Since flexible loads are often consumers, for concerns of privacy it is desirable for this problem to have a distributed implementation. Technically, this distributed implementation manifests itself as a time varying convex optimization problem constrained by the QoS of each load. In the literature, a time invariant form of this problem without all of the necessary QoS metrics for the flexible loads is often considered. Moving to a more realistic setup introduces additional technical challenges, due to the problems' time-varying nature. In this work, we develop an algorithm to account for the challenges introduced when considering a time varying setup with appropriate QoS metrics.
△ Less
Submitted 29 June, 2020;
originally announced June 2020.
-
Privacy-Preserving Policy Synthesis in Markov Decision Processes
Authors:
Parham Gohari,
Matthew Hale,
Ufuk Topcu
Abstract:
In decision-making problems, the actions of an agent may reveal sensitive information that drives its decisions. For instance, a corporation's investment decisions may reveal its sensitive knowledge about market dynamics. To prevent this type of information leakage, we introduce a policy synthesis algorithm that protects the privacy of the transition probabilities in a Markov decision process. We…
▽ More
In decision-making problems, the actions of an agent may reveal sensitive information that drives its decisions. For instance, a corporation's investment decisions may reveal its sensitive knowledge about market dynamics. To prevent this type of information leakage, we introduce a policy synthesis algorithm that protects the privacy of the transition probabilities in a Markov decision process. We use differential privacy as the mathematical definition of privacy. The algorithm first perturbs the transition probabilities using a mechanism that provides differential privacy. Then, based on the privatized transition probabilities, we synthesize a policy using dynamic programming. Our main contribution is to bound the "cost of privacy," i.e., the difference between the expected total rewards with privacy and the expected total rewards without privacy. We also show that computing the cost of privacy has time complexity that is polynomial in the parameters of the problem. Moreover, we establish that the cost of privacy increases with the strength of differential privacy protections, and we quantify this increase. Finally, numerical experiments on two example environments validate the established relationship between the cost of privacy and the strength of data privacy protections.
△ Less
Submitted 16 April, 2020;
originally announced April 2020.
-
An Algorithm for Multi-Objective Multi-Agent Optimization
Authors:
Maude J. Blondin,
Matthew Hale
Abstract:
Multi-agent optimization problems with many objective functions have drawn much interest over the past two decades. Many works on the subject minimize the sum of objective functions, which implicitly carries a decision about the problem formulation. Indeed, it represents a special case of a multi-objective problem, in which all objectives are prioritized equally. To the best of our knowledge, mult…
▽ More
Multi-agent optimization problems with many objective functions have drawn much interest over the past two decades. Many works on the subject minimize the sum of objective functions, which implicitly carries a decision about the problem formulation. Indeed, it represents a special case of a multi-objective problem, in which all objectives are prioritized equally. To the best of our knowledge, multi-objective optimization applied to multi-agent systems remains largely unexplored. Therefore, we propose a distributed algorithm that allows the exploration of Pareto optimal solutions for the non-homogeneously weighted sum of objective functions. In the problems we consider, each agent has one objective function to minimize based on a gradient method. Agents update their decision variables by exchanging information with other agents in the network. Information exchanges are weighted by each agent's individual weights that encode the extent to which they prioritize other agents' objectives. This paper provides a proof of convergence, performance bounds, and explicit limits for the results of agents' computations. Simulation results with different sizes of networks demonstrate the efficiency of the proposed approach and how the choice of weights impacts the agents' final result.
△ Less
Submitted 3 March, 2020;
originally announced March 2020.
-
Differentially Private Controller Synthesis With Metric Temporal Logic Specifications
Authors:
Zhe Xu,
Kasra Yazdani,
Matthew T. Hale,
Ufuk Topcu
Abstract:
Privacy is an important concern in various multiagent systems in which data collected from the agents are sensitive. We propose a differentially private controller synthesis approach for multi-agent systems subject to high-level specifications expressed in metric temporal logic (MTL). We consider a setting where each agent sends data to a cloud (computing station) through a set of local hubs and t…
▽ More
Privacy is an important concern in various multiagent systems in which data collected from the agents are sensitive. We propose a differentially private controller synthesis approach for multi-agent systems subject to high-level specifications expressed in metric temporal logic (MTL). We consider a setting where each agent sends data to a cloud (computing station) through a set of local hubs and the cloud is responsible for computing the control inputs of the agents. Specifically, each agent adds privacy noise (e.g., Gaussian noise) point-wise in time to its own outputs before sharing them with a local hub. Each local hub runs a Kalman filter to estimate the state of the corresponding agent and periodically sends such state estimates to the cloud. The cloud computes the optimal inputs for each agent subject to an MTL specification. While guaranteeing differential privacy of each agent, the controller is also synthesized to ensure a probabilistic guarantee for satisfying the MTL specification.We provide an implementation of the proposed method on a simulation case study with two Baxter-On-Wheels robots as the agents.
△ Less
Submitted 29 September, 2019;
originally announced September 2019.
-
Error Bounds and Guidelines for Privacy Calibration in Differentially Private Kalman Filtering
Authors:
Kasra Yazdani,
Matthew Hale
Abstract:
Differential privacy has emerged as a formal framework for protecting sensitive information in control systems. One key feature is that it is immune to post-processing, which means that arbitrary post-hoc computations can be performed on privatized data without weakening differential privacy. It is therefore common to filter private data streams. To characterize this setup, in this paper we presen…
▽ More
Differential privacy has emerged as a formal framework for protecting sensitive information in control systems. One key feature is that it is immune to post-processing, which means that arbitrary post-hoc computations can be performed on privatized data without weakening differential privacy. It is therefore common to filter private data streams. To characterize this setup, in this paper we present error and entropy bounds for Kalman filtering differentially private state trajectories. We consider systems in which an output trajectory is privatized in order to protect the state trajectory that produced it. We provide bounds on a priori and a posteriori error and differential entropy of a Kalman filter which is processing the privatized output trajectories. Using the error bounds we develop, we then provide guidelines to calibrate privacy levels in order to keep filter error within pre-specified bounds. Simulation results are presented to demonstrate these developments.
△ Less
Submitted 23 September, 2019; v1 submitted 19 March, 2019;
originally announced March 2019.
-
Stability of Leaderless Resource Consumption Networks
Authors:
Sebastian F. Ruf,
Matthew T. Hale,
Talha Manzoor,
Abubakr Muhammad
Abstract:
In this paper, we study the global stability properties of a multi-agent model of natural resource consumption that balances ecological and social network components in determining the consumption behavior of a group of agents. The social network is assumed to be leaderless, a condition that ensures that no single node has a greater influence than any other node on the dynamics of the resource con…
▽ More
In this paper, we study the global stability properties of a multi-agent model of natural resource consumption that balances ecological and social network components in determining the consumption behavior of a group of agents. The social network is assumed to be leaderless, a condition that ensures that no single node has a greater influence than any other node on the dynamics of the resource consumption. It is shown that any network structure can be made leaderless by the social preferences of the agents. The ecological network component includes a quantification of each agent's environmental concern, which captures each individual agent's threshold for when a resource becomes scarce. We show that leaderlessness and a mild bound on agents' environmental concern are jointly sufficient for global asymptotic stability of the consumption network to a positive consumption value, indicating that appropriately configured networks can continuously consume a resource without driving its value to zero. The behavior of these leaderless resource consumption networks is verified in simulation.
△ Less
Submitted 11 April, 2018;
originally announced April 2018.
