Showing 1–46 of 46 results for author: Shamma, J

Higher-Order Uncoupled Learning Dynamics and Nash Equilibrium

Authors: Sarah A. Toonsi, Jeff S. Shamma

Abstract: We study learnability of mixed-strategy Nash Equilibrium (NE) in general finite games using higher-order replicator dynamics as well as classes of higher-order uncoupled heterogeneous dynamics. In higher-order uncoupled learning dynamics, players have no access to utilities of opponents (uncoupled) but are allowed to use auxiliary states to further process information (higher-order). We establish… ▽ More We study learnability of mixed-strategy Nash Equilibrium (NE) in general finite games using higher-order replicator dynamics as well as classes of higher-order uncoupled heterogeneous dynamics. In higher-order uncoupled learning dynamics, players have no access to utilities of opponents (uncoupled) but are allowed to use auxiliary states to further process information (higher-order). We establish a link between uncoupled learning and feedback stabilization with decentralized control. Using this association, we show that for any finite game with an isolated completely mixed-strategy NE, there exist higher-order uncoupled learning dynamics that lead (locally) to that NE. We further establish the lack of universality of learning dynamics by linking learning to the control theoretic concept of simultaneous stabilization. We construct two games such that any higher-order dynamics that learn the completely mixed-strategy NE of one of these games can never learn the completely mixed-strategy NE of the other. Next, motivated by imposing natural restrictions on allowable learning dynamics, we introduce the Asymptotic Best Response (ABR) property. Dynamics with the ABR property asymptotically learn a best response in environments that are asymptotically stationary. We show that the ABR property relates to an internal stability condition on higher-order learning dynamics. We provide conditions under which NE are compatible with the ABR property. Finally, we address learnability of mixed-strategy NE in the bandit setting using a bandit version of higher-order replicator dynamics. △ Less

Submitted 12 June, 2025; originally announced June 2025.

Passivity, No-Regret, and Convergent Learning in Contractive Games

Authors: Hassan Abdelraouf, Georgios Piliouras, Jeff S. Shamma

Abstract: We investigate the interplay between passivity, no-regret, and convergence in contractive games for various learning dynamic models and their higher-order variants. Our setting is continuous time. Building on prior work for replicator dynamics, we show that if learning dynamics satisfy a passivity condition between the payoff vector and the difference between its evolving strategy and any fixed st… ▽ More We investigate the interplay between passivity, no-regret, and convergence in contractive games for various learning dynamic models and their higher-order variants. Our setting is continuous time. Building on prior work for replicator dynamics, we show that if learning dynamics satisfy a passivity condition between the payoff vector and the difference between its evolving strategy and any fixed strategy, then it achieves finite regret. We then establish that the passivity condition holds for various learning dynamics and their higher-order variants. Consequentially, the higher-order variants can achieve convergence to Nash equilibrium in cases where their standard order counterparts cannot, while maintaining a finite regret property. We provide numerical examples to illustrate the lack of finite regret of different evolutionary dynamic models that violate the passivity property. We also examine the fragility of the finite regret property in the case of perturbed learning dynamics. Continuing with passivity, we establish another connection between finite regret and passivity, but with the related equilibrium-independent passivity property. Finally, we present a passivity-based classification of dynamic models according to the various passivity notions they satisfy, namely, incremental passivity, $δ$-passivity, and equilibrium-independent passivity. This passivity-based classification provides a framework to analyze the convergence of learning dynamic models in contractive games. △ Less

Submitted 28 March, 2025; originally announced March 2025.

Constrained Optimization From a Control Perspective via Feedback Linearization

Authors: Runyu, Zhang, Arvind Raghunathan, Jeff Shamma, Na Li

Abstract: Tools from control and dynamical systems have proven valuable for analyzing and developing optimization methods. In this paper, we establish rigorous theoretical foundations for using feedback linearization -- a well-established nonlinear control technique -- to solve constrained optimization problems. For equality-constrained optimization, we establish global convergence rates to first-order Karu… ▽ More Tools from control and dynamical systems have proven valuable for analyzing and developing optimization methods. In this paper, we establish rigorous theoretical foundations for using feedback linearization -- a well-established nonlinear control technique -- to solve constrained optimization problems. For equality-constrained optimization, we establish global convergence rates to first-order Karush-Kuhn-Tucker (KKT) points and uncover the close connection between the FL method and the Sequential Quadratic Programming (SQP) algorithm. Building on this relationship, we extend the FL approach to handle inequality-constrained problems. Furthermore, we introduce a momentum-accelerated feedback linearization algorithm that achieves faster convergence, and provides a rigorous convergence guarantee. △ Less

Submitted 16 March, 2025; originally announced March 2025.

Distributed Learning Dynamics Converging to the Core of $B$-Matchings

Authors: Aya Hamed, Jeff S. Shamma

Abstract: $B$-Matching is a special case of matching problems where nodes can join multiple matchings with the degree of each node constrained by an upper bound, the node's $B$-value. The core solution of a bipartite $B… ▽ More $B$-Matching is a special case of matching problems where nodes can join multiple matchings with the degree of each node constrained by an upper bound, the node's $B$-value. The core solution of a bipartite $B$-matching is both a matching between the nodes respecting the upper bound constraint and an allocation of the weights of the edges among the nodes such that no group of nodes can deviate and collectively gain higher allocation. We present two learning dynamics that converge to the core of the bipartite $B$-matching problems. The first dynamics are centralized dynamics in the nature of the Hungarian method, which converge to the core in a polynomial time. The second dynamics are distributed dynamics, which converge to the core with probability one. For the distributed dynamics, a node maintains only a state consisting of (i) the aspiration levels for all of its possible matches and (ii) the matches, if any, to which it belongs. The node does not keep track of its history nor is it aware of the environment state. In each stage, a randomly activated node proposes to form a new match and changes its aspiration based on the success or failure of its proposal. At this stage, the proposing node inquires about the aspiration of the node it wants to match with to calculate the feasibility of the match. The environment matching structure changes whenever a proposal succeeds. A state is absorbing for the distributed dynamics if and only if it is in the core of the $B$-matching. △ Less

Submitted 16 September, 2024; v1 submitted 12 September, 2024; originally announced September 2024.

Comments: 10 pages, 5 figures; accepted for CDC 2024

Equilibrium Selection for Multi-agent Reinforcement Learning: A Unified Framework

Authors: Runyu Zhang, Jeff Shamma, Na Li

Abstract: While there are numerous works in multi-agent reinforcement learning (MARL), most of them focus on designing algorithms and proving convergence to a Nash equilibrium (NE) or other equilibrium such as coarse correlated equilibrium. However, NEs can be non-unique and their performance varies drastically. Thus, it is important to design algorithms that converge to Nash equilibrium with better rewards… ▽ More While there are numerous works in multi-agent reinforcement learning (MARL), most of them focus on designing algorithms and proving convergence to a Nash equilibrium (NE) or other equilibrium such as coarse correlated equilibrium. However, NEs can be non-unique and their performance varies drastically. Thus, it is important to design algorithms that converge to Nash equilibrium with better rewards or social welfare. In contrast, classical game theory literature has extensively studied equilibrium selection for multi-agent learning in normal-form games, demonstrating that decentralized learning algorithms can asymptotically converge to potential-maximizing or Pareto-optimal NEs. These insights motivate this paper to investigate equilibrium selection in the MARL setting. We focus on the stochastic game model, leveraging classical equilibrium selection results from normal-form games to propose a unified framework for equilibrium selection in stochastic games. The proposed framework is highly modular and can extend various learning rules and their corresponding equilibrium selection results from normal-form games to the stochastic game setting. △ Less

Submitted 13 June, 2024; originally announced June 2024.

Polynomial Lyapunov Functions and Invariant Sets from a New Hierarchy of Quadratic Lyapunov Functions for LTV Systems

Authors: Hassan Abdelraouf, Eric Feron, Jeff S. Shamma

Abstract: We introduce a new class of quadratic functions based on a hierarchy of linear time-varying (LTV) dynamical systems. These quadratic functions in the higher order space can be also seen as a non-homogeneous polynomial Lyapunov functions for the original system, i.e the first system in the hierarchy. These non-homogeneous polynomials are used to obtain accurate outer approximation for the reachable… ▽ More We introduce a new class of quadratic functions based on a hierarchy of linear time-varying (LTV) dynamical systems. These quadratic functions in the higher order space can be also seen as a non-homogeneous polynomial Lyapunov functions for the original system, i.e the first system in the hierarchy. These non-homogeneous polynomials are used to obtain accurate outer approximation for the reachable set given the initial condition and less conservative bounds for the impulse response peak of linear, possibly time-varying systems. In addition, we pose an extension to the presented approach to construct invariant sets that are not necessarily Lyapunov functions. The introduced methods are based on elementary linear systems theory and offer very much flexibility in defining arbitrary polynomial Lyapunov functions and invariant sets for LTV systems. △ Less

Submitted 23 January, 2024; originally announced January 2024.

Distributed Learning Dynamics for Coalitional Games

Authors: Aya Hamed, Jeff S. Shamma

Abstract: In the framework of transferable utility coalitional games, a scoring (characteristic) function determines the value of any subset/coalition of agents. Agents decide on both which coalitions to form and the allocations of the values of the formed coalitions among their members. An important concept in coalitional games is that of a core solution, which is a partitioning of agents into coalitions a… ▽ More In the framework of transferable utility coalitional games, a scoring (characteristic) function determines the value of any subset/coalition of agents. Agents decide on both which coalitions to form and the allocations of the values of the formed coalitions among their members. An important concept in coalitional games is that of a core solution, which is a partitioning of agents into coalitions and an associated allocation to each agent under which no group of agents can get a higher allocation by forming an alternative coalition. We present distributed learning dynamics for coalitional games that converge to a core solution whenever one exists. In these dynamics, an agent maintains a state consisting of (i) an aspiration level for its allocation and (ii) the coalition, if any, to which it belongs. In each stage, a randomly activated agent proposes to form a new coalition and changes its aspiration based on the success or failure of its proposal. The coalition membership structure is changed, accordingly, whenever the proposal succeeds. Required communications are that: (i) agents in the proposed new coalition need to reveal their current aspirations to the proposing agent, and (ii) agents are informed if they are joining the proposed coalition or if their existing coalition is broken. The proposing agent computes the feasibility of forming the coalition. We show that the dynamics hit an absorbing state whenever a core solution is reached. We further illustrate the distributed learning dynamics on a multi-agent task allocation setting. △ Less

Submitted 27 November, 2023; v1 submitted 13 September, 2023; originally announced September 2023.

Comments: 9 pages, 5 figures; accepted for CDC 2023

Non-asymptotic System Identification for Linear Systems with Nonlinear Policies

Authors: Yingying Li, Tianpeng Zhang, Subhro Das, Jeff Shamma, Na Li

Abstract: This paper considers a single-trajectory system identification problem for linear systems under general nonlinear and/or time-varying policies with i.i.d. random excitation noises. The problem is motivated by safe learning-based control for constrained linear systems, where the safe policies during the learning process are usually nonlinear and time-varying for satisfying the state and input const… ▽ More This paper considers a single-trajectory system identification problem for linear systems under general nonlinear and/or time-varying policies with i.i.d. random excitation noises. The problem is motivated by safe learning-based control for constrained linear systems, where the safe policies during the learning process are usually nonlinear and time-varying for satisfying the state and input constraints. In this paper, we provide a non-asymptotic error bound for least square estimation when the data trajectory is generated by any nonlinear and/or time-varying policies as long as the generated state and action trajectories are bounded. This significantly generalizes the existing non-asymptotic guarantees for linear system identification, which usually consider i.i.d. random inputs or linear policies. Interestingly, our error bound is consistent with that for linear policies with respect to the dependence on the trajectory length, system dimensions, and excitation levels. Lastly, we demonstrate the applications of our results by safe learning with robust model predictive control and provide numerical analysis. △ Less

Submitted 17 June, 2023; originally announced June 2023.

Higher-Order Uncoupled Dynamics Do Not Lead to Nash Equilibrium -- Except When They Do

Authors: Sarah A. Toonsi, Jeff S. Shamma

Abstract: The framework of multi-agent learning explores the dynamics of how individual agent strategies evolve in response to the evolving strategies of other agents. Of particular interest is whether or not agent strategies converge to well known solution concepts such as Nash Equilibrium (NE). Most "fixed order" learning dynamics restrict an agent's underlying state to be its own strategy. In "higher ord… ▽ More The framework of multi-agent learning explores the dynamics of how individual agent strategies evolve in response to the evolving strategies of other agents. Of particular interest is whether or not agent strategies converge to well known solution concepts such as Nash Equilibrium (NE). Most "fixed order" learning dynamics restrict an agent's underlying state to be its own strategy. In "higher order" learning, agent dynamics can include auxiliary states that can capture phenomena such as path dependencies. We introduce higher-order gradient play dynamics that resemble projected gradient ascent with auxiliary states. The dynamics are "payoff based" in that each agent's dynamics depend on its own evolving payoff. While these payoffs depend on the strategies of other agents in a game setting, agent dynamics do not depend explicitly on the nature of the game or the strategies of other agents. In this sense, dynamics are "uncoupled" since an agent's dynamics do not depend explicitly on the utility functions of other agents. We first show that for any specific game with an isolated completely mixed-strategy NE, there exist higher-order gradient play dynamics that lead (locally) to that NE, both for the specific game and nearby games with perturbed utility functions. Conversely, we show that for any higher-order gradient play dynamics, there exists a game with a unique isolated completely mixed-strategy NE for which the dynamics do not lead to NE. These results build on prior work that showed that uncoupled fixed-order learning cannot lead to NE in certain instances, whereas higher-order variants can. Finally, we consider the mixed-strategy equilibrium associated with coordination games. While higher-order gradient play can converge to such equilibria, we show such dynamics must be inherently internally unstable. △ Less

Submitted 19 November, 2023; v1 submitted 9 April, 2023; originally announced April 2023.

Algebraic Lyapunov Functions for Homogeneous Dynamic Systems

Authors: Hassan Abdelraouf, Eric Feron, Jeff Shamma

Abstract: A method for constructing homogeneous Lyapunov functions of degree 1 from polynomial invariant sets is presented for linear time varying systems, homogeneous dynamic systems and the class of nonlinear systems that can be represented as such. The method allows the development of Lyapunov functions that are not necessarily symmetric about the origin, unlike the polynomial, homogeneous Lyapunov funct… ▽ More A method for constructing homogeneous Lyapunov functions of degree 1 from polynomial invariant sets is presented for linear time varying systems, homogeneous dynamic systems and the class of nonlinear systems that can be represented as such. The method allows the development of Lyapunov functions that are not necessarily symmetric about the origin, unlike the polynomial, homogeneous Lyapunov functions discussed in the prior literature. The work is illustrated by very simple examples. △ Less

Submitted 3 March, 2023; originally announced March 2023.

Online switching control with stability and regret guarantees

Authors: Yingying Li, James A. Preiss, Na Li, Yiheng Lin, Adam Wierman, Jeff Shamma

Abstract: This paper considers online switching control with a finite candidate controller pool, an unknown dynamical system, and unknown cost functions. The candidate controllers can be unstabilizing policies. We only require at least one candidate controller to satisfy certain stability properties, but we do not know which one is stabilizing. We design an online algorithm that guarantees finite-gain stabi… ▽ More This paper considers online switching control with a finite candidate controller pool, an unknown dynamical system, and unknown cost functions. The candidate controllers can be unstabilizing policies. We only require at least one candidate controller to satisfy certain stability properties, but we do not know which one is stabilizing. We design an online algorithm that guarantees finite-gain stability throughout the duration of its execution. We also provide a sublinear policy regret guarantee compared with the optimal stabilizing candidate controller. Lastly, we numerically test our algorithm on quadrotor planar flights and compare it with a classical switching control algorithm, falsification-based switching, and a classical multi-armed bandit algorithm, Exp3 with batches. △ Less

Submitted 23 January, 2023; v1 submitted 20 January, 2023; originally announced January 2023.

Can Competition Outperform Collaboration? The Role of Misbehaving Agents

Authors: Luca Ballotta, Giacomo Como, Jeff S. Shamma, Luca Schenato

Abstract: We investigate a novel approach to resilient distributed optimization with quadratic costs in a multi-agent system prone to unexpected events that make some agents misbehave. In contrast to commonly adopted filtering strategies, we draw inspiration from phenomena modeled through the Friedkin-Johnsen dynamics and argue that adding competition to the mix can improve resilience in the presence of mis… ▽ More We investigate a novel approach to resilient distributed optimization with quadratic costs in a multi-agent system prone to unexpected events that make some agents misbehave. In contrast to commonly adopted filtering strategies, we draw inspiration from phenomena modeled through the Friedkin-Johnsen dynamics and argue that adding competition to the mix can improve resilience in the presence of misbehaving agents. Our intuition is corroborated by analytical and numerical results showing that (i) there exists a nontrivial trade-off between full collaboration and full competition and (ii) our competition-based approach can outperform state-of-the-art algorithms based on Weighted Mean Subsequence Reduced. We also study impact of communication topology and connectivity on resilience, pointing out insights to robust network design. △ Less

Submitted 30 October, 2023; v1 submitted 4 July, 2022; originally announced July 2022.

Comments: Accepted in IEEE TAC; 17 pages, 44 figures

MSC Class: 93D50 (Primary) 93B70 (Secondary) ACM Class: I.2.8; I.2.9

Journal ref: IEEE Transactions on Automatic Control, vol. 69, no. 4, pp. 2308-2323, 2024

Competition-Based Resilience in Distributed Quadratic Optimization

Authors: Luca Ballotta, Giacomo Como, Jeff S. Shamma, Luca Schenato

Abstract: This paper proposes a novel approach to resilient distributed optimization with quadratic costs in a networked control system (e.g., wireless sensor network, power grid, robotic team) prone to external attacks (e.g., hacking, power outage) that cause agents to misbehave. Departing from classical filtering strategies proposed in literature, we draw inspiration from a game-theoretic formulation of t… ▽ More This paper proposes a novel approach to resilient distributed optimization with quadratic costs in a networked control system (e.g., wireless sensor network, power grid, robotic team) prone to external attacks (e.g., hacking, power outage) that cause agents to misbehave. Departing from classical filtering strategies proposed in literature, we draw inspiration from a game-theoretic formulation of the consensus problem and argue that adding competition to the mix can enhance resilience in the presence of malicious agents. Our intuition is corroborated by analytical and numerical results showing that i) our strategy highlights the presence of a nontrivial tradeoff between blind collaboration and full competition, and ii) such competition-based approach can outperform state-of-the-art algorithms based on Mean Subsequence Reduced. △ Less

Submitted 10 January, 2024; v1 submitted 26 March, 2022; originally announced March 2022.

Comments: 7 pages, 8 figures; accepted for CDC 2022

MSC Class: 93B70 (Primary) 68M18; 93D09; 93D50 (Secondary) ACM Class: I.2.11

Journal ref: Proceedings of the IEEE Conference on Decision and Control, pp. 6454-6459, 2022

Multi-sided Matching for the Association of Space-Air-Ground Integrated Systems

Authors: Abdoul Karim A. H. Saliah, Doha Hamza, Hajar El Hammouti, Jeff S Shamma, Mohamed-Slim Alouini

Abstract: Space-air-ground integrated networks (SAGINs) will play a pivotal role in 6G communication systems. They are considered a promising technology for enhancing network capacity in densely populated urban areas and extending connectivity to rural regions. However, the complex, multi-layered, and heterogeneous nature of SAGINs demands an innovative approach to designing their multi-tier associations. I… ▽ More Space-air-ground integrated networks (SAGINs) will play a pivotal role in 6G communication systems. They are considered a promising technology for enhancing network capacity in densely populated urban areas and extending connectivity to rural regions. However, the complex, multi-layered, and heterogeneous nature of SAGINs demands an innovative approach to designing their multi-tier associations. In this context, we propose a modeling of the SAGINs association problem using multi-sided matching theory. Our objective is to devise a reliable, asynchronous, and fully distributed approach that associates nodes across the layers to maximize the total end-to-end rate of the assigned agents. To achieve this, our problem is formulated as a multi-sided many-to-one matching game. We introduce a randomized matching algorithm with minimal information exchange. The algorithm is shown to reach an efficient and stable association between nodes in adjacent layers. Simulation results show that our proposed approach yields significant gains compared to both greedy and distance-based algorithms. △ Less

Submitted 11 July, 2024; v1 submitted 19 October, 2021; originally announced November 2021.

Comments: Accepted at IEEE VTC-Spring 2024

Safe Adaptive Learning-based Control for Constrained Linear Quadratic Regulators with Regret Guarantees

Authors: Yingying Li, Subhro Das, Jeff Shamma, Na Li

Abstract: We study the adaptive control of an unknown linear system with a quadratic cost function subject to safety constraints on both the states and actions. The challenges of this problem arise from the tension among safety, exploration, performance, and computation. To address these challenges, we propose a polynomial-time algorithm that guarantees feasibility and constraint satisfaction with high prob… ▽ More We study the adaptive control of an unknown linear system with a quadratic cost function subject to safety constraints on both the states and actions. The challenges of this problem arise from the tension among safety, exploration, performance, and computation. To address these challenges, we propose a polynomial-time algorithm that guarantees feasibility and constraint satisfaction with high probability under proper conditions. Our algorithm is implemented on a single trajectory and does not require system restarts. Further, we analyze the regret of our learning algorithm compared to the optimal safe linear controller with known model information. The proposed algorithm can achieve a $\tilde O(T^{2/3})$ regret, where $T$ is the number of stages and $\tilde O(\cdot)$ absorbs some logarithmic terms of $T$. △ Less

Submitted 31 October, 2021; originally announced November 2021.

Point-to-Point Communication in Integrated Satellite-Aerial Networks: State-of-the-art and Future Challenges

Authors: Nasir Saeed, Heba Almorad, Hayssam Dahrouj, Tareq Y. Al-Naffouri, Jeff S. Shamma, Mohamed-Slim Alouini

Abstract: This paper overviews point-to-point (P2P) links for integrated satellite-aerial networks, which are envisioned to be among the key enablers of the sixth-generation (6G) of wireless networks vision. The paper first outlines the unique characteristics of such integrated large-scale complex networks, often denoted by spatial networks, and focuses on two particular space-air infrastructures, namely, s… ▽ More This paper overviews point-to-point (P2P) links for integrated satellite-aerial networks, which are envisioned to be among the key enablers of the sixth-generation (6G) of wireless networks vision. The paper first outlines the unique characteristics of such integrated large-scale complex networks, often denoted by spatial networks, and focuses on two particular space-air infrastructures, namely, satellites networks and high-altitude platforms (HAPs). The paper then classifies the connecting P2P communications links as satellite-to-satellite links at the same layer (SSLL), satellite-to-satellite links at different layers (SSLD), and HAP-to-HAP links (HHL). The paper overviews each layer of such spatial networks separately, and highlights the possible natures of the connecting links (i.e., radio-frequency or free-space optics) with a dedicated overview to the existing link-budget results. The paper, afterwards, presents the prospective merit of realizing such an integrated satellite-HAP network towards providing broadband services in under-served and remote areas. Finally, the paper sheds light on several future research directions in the context of spatial networks, namely large-scale network optimization, intelligent offloading, smart platforms, energy efficiency, multiple access schemes, and distributed spatial networks. △ Less

Submitted 11 December, 2020; originally announced December 2020.

Comments: 17 pages

RISCuer: A Reliable Multi-UAV Search and Rescue Testbed

Authors: Mohamed Abdelkader, Usman A. Fiaz, Noureddine Toumi, Mohamed A. Mabrok, Jeff S. Shamma

Abstract: We present the Robotics Intelligent Systems & Control (RISC) Lab multiagent testbed for reliable search and rescue and aerial transport in outdoor environments. The system consists of a team of three multirotor unmanned aerial vehicles (UAVs), which are capable of autonomously searching, picking up, and transporting randomly distributed objects in an outdoor field. The method involves vision based… ▽ More We present the Robotics Intelligent Systems & Control (RISC) Lab multiagent testbed for reliable search and rescue and aerial transport in outdoor environments. The system consists of a team of three multirotor unmanned aerial vehicles (UAVs), which are capable of autonomously searching, picking up, and transporting randomly distributed objects in an outdoor field. The method involves vision based object detection and localization, passive aerial grasping with our novel design, GPS based UAV navigation, and safe release of the objects at the drop zone. Our cooperative strategy ensures safe spatial separation between UAVs at all times and we prevent any conflicts at the drop zone using communication enabled consensus. All computation is performed onboard each UAV. We describe the complete software and hardware architecture for the system and demonstrate its reliable performance using comprehensive outdoor experiments, and by comparing our results with some recent, similar works. △ Less

Submitted 7 December, 2021; v1 submitted 12 June, 2020; originally announced June 2020.

Comments: Book chapter: 41 pages, 27 figures (Minor revision: Corrected references)

Journal ref: Unmanned Aerial Systems, 2021

The Optimal and the Greedy: Drone Association and Positioning Schemes for Internet of UAVs

Authors: Hajar El Hammouti, Doha Hamza, Basem Shihada, Mohamed-Slim Alouini, Jeff S. Shamma

Abstract: This work considers the deployment of unmanned aerial vehicles (UAVs) over a predefined area to serve a number of ground users. Due to the heterogeneous nature of the network,the UAVs may cause severe interference to the transmissions of each other. Hence, a judicious design of the user-UAV association and UAV locations is desired. A potential game is defined where the players are the UAVs. The po… ▽ More This work considers the deployment of unmanned aerial vehicles (UAVs) over a predefined area to serve a number of ground users. Due to the heterogeneous nature of the network,the UAVs may cause severe interference to the transmissions of each other. Hence, a judicious design of the user-UAV association and UAV locations is desired. A potential game is defined where the players are the UAVs. The potential function is the total sum-rate of the users. The agents utility in the potential games is their marginal contribution to the global welfare or their so-called wonderful life utility. A game-theoretic learning algorithm, binary log-linear learning (BLLL), is then applied to the problem. Given the potential game structure, a consequence of our utility design, the stochastically stable states using BLLL are guaranteed to be the potential maximizers. Hence, we optimally solve the user-UAV association and 3D-location problem. Next, we exploit the sub-modular features of the sum rate function for a given configuration of UAVs to design an efficient greedy algorithm. Despite the simplicity of the greedy algorithm, it comes with a guaranteed performance of $1-1/e$ of the optimal solution. To further reduce the number of iterations, we propose another heuristic greedy algorithm that provides very good results. Our simulations show that, in practice, the proposed greedy approaches achieve significant performance in a few number of iterations. △ Less

Submitted 2 April, 2020; originally announced April 2020.

Comments: Submitted to IEEE Journal of Internet Of Things

Network Effects on Robustness of Dynamic Systems

Authors: Ketan Savla, Jeff S. Shamma, Munther A. Dahleh

Abstract: We review selected results related to robustness of networked systems in finite and asymptotically large size regimes, under static and dynamical settings. In the static setting, within the framework of flow over finite networks, we discuss the effect of physical constraints on robustness to loss in link capacities. In the dynamical setting, we review several settings in which small gain type anal… ▽ More We review selected results related to robustness of networked systems in finite and asymptotically large size regimes, under static and dynamical settings. In the static setting, within the framework of flow over finite networks, we discuss the effect of physical constraints on robustness to loss in link capacities. In the dynamical setting, we review several settings in which small gain type analysis provides tight robustness guarantees for linear dynamics over finite networks towards worst-case and stochastic disturbances. We also discuss network flow dynamic settings where nonlinear techniques facilitate in understanding the effect on robustness of constraints on capacity and information, substituting information with control action, and cascading failure. We also contrast the latter with a representative contagion model. For asymptotically large networks, we discuss the role of network properties in connecting microscopic shocks to emergent macroscopic fluctuations under linear dynamics as well as for economic networks at equilibrium. Through the review of these results, the paper aims to achieve two objectives. First, to highlight selected settings in which the role of interconnectivity structure of a network on its robustness is well-understood. Second, to highlight a few additional settings in which existing system theoretic tools give tight robustness guarantees, and which are also appropriate avenues for future network-theoretic investigations. △ Less

Submitted 13 September, 2019; originally announced September 2019.

Comments: To appear in Annual Review of Control, Robotics, and Autonomous Systems

Smart Jammer and LTE Network Strategies in An Infinite-Horizon Zero-Sum Repeated Game with Asymmetric and Incomplete Information

Authors: Farhan M. Aziz, Lichun Li, Jeff S. Shamma, Gordon L. Stuber

Abstract: LTE/LTE-Advanced networks are known to be vulnerable to denial-of-service and loss-of-service attacks from smart jammers. In this article, the interaction between a smart jammer and LTE network is modeled as an infinite-horizon, zero-sum, asymmetric repeated game. The smart jammer and eNode B are modeled as the informed and the uninformed player, respectively. The main purpose of this article is t… ▽ More LTE/LTE-Advanced networks are known to be vulnerable to denial-of-service and loss-of-service attacks from smart jammers. In this article, the interaction between a smart jammer and LTE network is modeled as an infinite-horizon, zero-sum, asymmetric repeated game. The smart jammer and eNode B are modeled as the informed and the uninformed player, respectively. The main purpose of this article is to construct efficient suboptimal strategies for both players that can be used to solve the above-mentioned infinite-horizon repeated game with asymmetric and incomplete information. It has been shown in game-theoretic literature that security strategies provide optimal solution in zero-sum games. It is also shown that both players' security strategies in an infinite-horizon asymmetric game depend only on the history of the informed player's actions. However, fixed-sized sufficient statistics are needed for both players to solve the above-mentioned game efficiently. The smart jammer uses its evolving belief state as the fixed-sized sufficient statistics for the repeated game. Whereas, the LTE network (uninformed player) uses worst-case regret of its security strategy and its anti-discounted update as the fixed-sized sufficient statistics. Although fixed-sized sufficient statistics are employed by both players, optimal security strategy computation in λ-discounted asymmetric games is still hard to perform because of non-convexity. Hence, the problem is convexified in this article by devising `approximated' security strategies for both players that are based on approximated optimal game value. However, `approximated' strategies require full monitoring. Therefore, a simplistic yet effective `expected' strategy is also constructed for the LTE network that does not require full monitoring. The simulation results show that the smart jammer plays non-revealing and misleading strategies. △ Less

Submitted 25 April, 2019; originally announced April 2019.

Herdability of Linear Systems Based on Sign Patterns and Graph Structures

Authors: Sebastian F Ruf, Magnus Egerstedt, Jeff S. Shamma

Abstract: We consider the notion of herdability, a set-based reachability condition, which asks whether the state of a system can be controlled to be element-wise larger than a non-negative threshold. First a number of foundational results on herdability of a continuous time, linear time invariant system are presented. These show that the herdability of a linear system can be determined based on certain mat… ▽ More We consider the notion of herdability, a set-based reachability condition, which asks whether the state of a system can be controlled to be element-wise larger than a non-negative threshold. First a number of foundational results on herdability of a continuous time, linear time invariant system are presented. These show that the herdability of a linear system can be determined based on certain matrices, such as the controllability matrix, which arise in the study of controllability of linear systems. Second, the relationship between the sign pattern of the underlying graph structure of a system and the herdability properties of the system is investigated. In doing so the notion of sign herdability is introduced which captures classes of systems whose sign pattern determines their herdability. We identify a set of conditions, first on the sign pattern of the controllability matrix and then on the underlying graph structure, that ensure that the system is sign herdable. △ Less

Submitted 16 April, 2019; originally announced April 2019.

Comments: arXiv admin note: text overlap with arXiv:1804.04230

Payoff Dynamics Model and Evolutionary Dynamics Model: Feedback and Convergence to Equilibria

Authors: Shinkyu Park, Nuno C. Martins, Jeff S. Shamma

Abstract: This tutorial article puts forth a framework to analyze the noncooperative strategic interactions among the members of a large population of bounded rationality agents. Our approach hinges on, unifies and generalizes existing methods and models predicated in evolutionary and population games. It does so by adopting a system-theoretic formalism that is well-suited for a broad engineering audience f… ▽ More This tutorial article puts forth a framework to analyze the noncooperative strategic interactions among the members of a large population of bounded rationality agents. Our approach hinges on, unifies and generalizes existing methods and models predicated in evolutionary and population games. It does so by adopting a system-theoretic formalism that is well-suited for a broad engineering audience familiar with the basic tenets of nonlinear dynamical systems, Lyapunov stability, storage functions, and passivity. The framework is pertinent for engineering applications in which a large number of agents have the authority to select and repeatedly revise their strategies. A mechanism that is inherent to the problem at hand or is designed and implemented by a coordinator ascribes a payoff to each possible strategy. Typically, the agents will prioritize switching to strategies whose payoff is either higher than the current one or exceeds the population average. The article puts forth a systematic methodology to characterize the stability of the dynamical system that results from the feedback interaction between the payoff mechanism and the revision process. This is important because the set of stable equilibria is an accurate predictor of the population's long-term behavior. The article includes rigorous proofs and examples of application of the stability results, which also extend the state of the art because, unlike previously published work, they allow for a rather general class of dynamical payoff mechanisms. The new results and concepts proposed here are thoroughly compared to previous work, methods and applications of evolutionary and population games. △ Less

Submitted 14 July, 2020; v1 submitted 5 March, 2019; originally announced March 2019.

usBot: A Modular Robotic Testbed for Programmable Self-Assembly

Authors: Usman A. Fiaz, Jeff S. Shamma

Abstract: We present the design, characterization, and experimental results for a new modular robotic system for programmable self-assembly. The proposed system uses the Hybrid Cube Model (HCM), which integrates classical features from both deterministic and stochastic self-organization models. Thus, for instance, the modules are passive as far as their locomotion is concerned (stochastic), and yet they pos… ▽ More We present the design, characterization, and experimental results for a new modular robotic system for programmable self-assembly. The proposed system uses the Hybrid Cube Model (HCM), which integrates classical features from both deterministic and stochastic self-organization models. Thus, for instance, the modules are passive as far as their locomotion is concerned (stochastic), and yet they possess an active undocking routine (deterministic). The robots are constructed entirely from readily accessible components and unlike many existing robots, their excitation is not fluid mediated. Instead, the actuation setup is a solid state, independently programmable and highly portable platform. The system is capable of demonstrating fully autonomous and distributed stochastic self-assembly in two dimensions. It is shown to emulate the performance of several existing modular systems and promises to be a substantial effort towards developing a universal testbed for programmable self-assembly. △ Less

Submitted 29 June, 2019; v1 submitted 5 January, 2019; originally announced January 2019.

Comments: Accepted as a conference paper at 2019 IFAC Joint MECHATRONICS and NOLCOS

Infrastructure-free Localization of Aerial Robots with Ultrawideband Sensors

Authors: Samet Guler, Mohamed Abdelkader, Jeff S. Shamma

Abstract: Robots in a swarm take advantage of a motion capture system or GPS sensors to obtain their global position. However, motion capture systems are environment-dependent and GPS sensors are not reliable in occluded environments. For a reliable and versatile operation in a swarm, robots must sense each other and interact locally. Motivated by this requirement, here we propose an on-board localization f… ▽ More Robots in a swarm take advantage of a motion capture system or GPS sensors to obtain their global position. However, motion capture systems are environment-dependent and GPS sensors are not reliable in occluded environments. For a reliable and versatile operation in a swarm, robots must sense each other and interact locally. Motivated by this requirement, here we propose an on-board localization framework for multi-robot systems. Our framework consists of an anchor robot with three ultrawideband (UWB) sensors and a tag robot with a single UWB sensor. The anchor robot utilizes the three UWB sensors as a localization infrastructure and estimates the tag robot's location by using its on-board sensing and computational capabilities solely, without explicit inter-robot communication. We utilize a dual Monte-Carlo localization approach to capture the agile maneuvers of the tag robot with an acceptable precision. We validate the effectiveness of our algorithm with simulations and indoor and outdoor experiments on a two-drone setup. The proposed dual MCL algorithm yields highly accurate estimates for various speed profiles of the tag robot and demonstrates a superior performance over the standard particle filter and the extended Kalman Filter. △ Less

Submitted 21 September, 2018; originally announced September 2018.

Comments: 14 pages

Herding Positive, Complex Networks

Authors: Sebastian F. Ruf, Magnus Egersted, Jeff S. Shamma

Abstract: The problem of controlling complex networks is of interest to disciplines ranging from biology to swarm robotics. However, controllability can be too strict a condition, failing to capture a range of desirable behaviors. Herdability, which describes the ability to drive a system to a specific set in the state space, was recently introduced as an alternative network control notion. This paper consi… ▽ More The problem of controlling complex networks is of interest to disciplines ranging from biology to swarm robotics. However, controllability can be too strict a condition, failing to capture a range of desirable behaviors. Herdability, which describes the ability to drive a system to a specific set in the state space, was recently introduced as an alternative network control notion. This paper considers the application of herdability to the study of complex networks under the assumption that a positive system evolves on the network. The herdability of a class of networked systems is investigated and two problems related to ensuring system herdability are explored. The first is the input addition problem, which investigates which nodes in a network should receive inputs to ensure that the system is herdable. The second is a related problem of selecting the best single node from which to herd the network, in the case that a single node is guaranteed to make the system is herdable. In order to select the best herding node, a novel control energy based herdability centrality measure is introduced. △ Less

Submitted 28 April, 2018; v1 submitted 12 April, 2018; originally announced April 2018.

Comments: Updated the proof of Theorem 2

Herdable Systems Over Signed, Directed Graphs

Authors: Sebastian F. Ruf, Magnus Egerstedt, Jeff S. Shamma

Abstract: This paper considers the notion of herdability, a set-based reachability condition, which asks whether the state of a system can be controlled to be element-wise larger than a non-negative threshold. The basic theory of herdable systems is presented, including a necessary and sufficient condition for herdability. This paper then considers the impact of the underlying graph structure of a linear sy… ▽ More This paper considers the notion of herdability, a set-based reachability condition, which asks whether the state of a system can be controlled to be element-wise larger than a non-negative threshold. The basic theory of herdable systems is presented, including a necessary and sufficient condition for herdability. This paper then considers the impact of the underlying graph structure of a linear system on the herdability of the system, for the case where the graph is represented as signed and directed. By classifying nodes based on the length and sign of walks from an input, we find a class of completely herdable systems as well as provide a complete characterization of nodes that can be herded in systems with an underlying graph that is a directed out-branching rooted at a single input. △ Less

Submitted 11 April, 2018; originally announced April 2018.

Comments: To Appear in the proceedings of the American Control Conference 2018

Path to Stochastic Stability: Comparative Analysis of Stochastic Learning Dynamics in Games

Authors: Hassan Jaleel, Jeff S. Shamma

Abstract: Stochastic stability is a popular solution concept for stochastic learning dynamics in games. However, a critical limitation of this solution concept is its inability to distinguish between different learning rules that lead to the same steady-state behavior. We address this limitation for the first time and develop a framework for the comparative analysis of stochastic learning dynamics with diff… ▽ More Stochastic stability is a popular solution concept for stochastic learning dynamics in games. However, a critical limitation of this solution concept is its inability to distinguish between different learning rules that lead to the same steady-state behavior. We address this limitation for the first time and develop a framework for the comparative analysis of stochastic learning dynamics with different update rules but same steady-state behavior. We present the framework in the context of two learning dynamics: Log-Linear Learning (LLL) and Metropolis Learning (ML). Although both of these dynamics have the same stochastically stable states, LLL and ML correspond to different behavioral models for decision making. Moreover, we demonstrate through an example setup of sensor coverage game that for each of these dynamics, the paths to stochastically stable states exhibit distinctive behaviors. Therefore, we propose multiple criteria to analyze and quantify the differences in the short and medium run behavior of stochastic learning dynamics. We derive and compare upper bounds on the expected hitting time to the set of Nash equilibria for both LLL and ML. For the medium to long-run behavior, we identify a set of tools from the theory of perturbed Markov chains that result in a hierarchical decomposition of the state space into collections of states called cycles. We compare LLL and ML based on the proposed criteria and develop invaluable insights into the comparative behavior of the two dynamics. △ Less

Submitted 8 April, 2018; originally announced April 2018.

Passivity and Evolutionary Game Dynamics

Authors: Shinkyu Park, Jeff S. Shamma, Nuno C. Martins

Abstract: This paper investigates an energy conservation and dissipation -- passivity -- aspect of dynamic models in evolutionary game theory. We define a notion of passivity using the state-space representation of the models, and we devise systematic methods to examine passivity and to identify properties of passive dynamic models. Based on the methods, we describe how passivity is connected to stability i… ▽ More This paper investigates an energy conservation and dissipation -- passivity -- aspect of dynamic models in evolutionary game theory. We define a notion of passivity using the state-space representation of the models, and we devise systematic methods to examine passivity and to identify properties of passive dynamic models. Based on the methods, we describe how passivity is connected to stability in population games and illustrate stability of passive dynamic models using numerical simulations. △ Less

Submitted 21 March, 2018; originally announced March 2018.

Security Strategies of Both Players in Asymmetric Information Zero-Sum Stochastic Games with an Informed Controller

Authors: Lichun Li, Cedric Langbort, Jeff S. Shamma

Abstract: This paper considers a zero-sum two-player asymmetric information stochastic game where only one player knows the system state, and the transition law is controlled by the informed player only. For the informed player, it has been shown that the security strategy only depends on the belief and the current stage. We provide LP formulations whose size is only linear in the size of the uninformed pla… ▽ More This paper considers a zero-sum two-player asymmetric information stochastic game where only one player knows the system state, and the transition law is controlled by the informed player only. For the informed player, it has been shown that the security strategy only depends on the belief and the current stage. We provide LP formulations whose size is only linear in the size of the uninformed player's action set to compute both history based and belief based security strategies. For the uninformed player, we focus on the regret, the difference between 0 and the future payoff guaranteed by the uninformed player in every possible state. Regret is a real vector of the same size as the belief, and depends only on the action of the informed player and the strategy of the uninformed player. This paper shows that the uninformed player has a security strategy that only depends on the regret and the current stage. LP formulations are then given to compute the history based security strategy, the regret at every stage, and the regret based security strategy. The size of the LP formulations are again linear in the size of the uninformed player action set. Finally, an intrusion detection problem is studied to demonstrate the main results in this paper. △ Less

Submitted 7 November, 2017; originally announced November 2017.

Comments: submitted to special issue in the journal Dynamic Games and Applications

Distributed Submodular Minimization And Motion Coordination Over Discrete State Space

Authors: Hassan Jaleel, Jeff Shamma

Abstract: We develop a framework for the distributed minimization of submodular functions. Submodular functions are a discrete analog of convex functions and are extensively used in large-scale combinatorial optimization problems. While there has been significant interest in the distributed formulations of convex optimization problems, distributed minimization of submodular functions has received relatively… ▽ More We develop a framework for the distributed minimization of submodular functions. Submodular functions are a discrete analog of convex functions and are extensively used in large-scale combinatorial optimization problems. While there has been significant interest in the distributed formulations of convex optimization problems, distributed minimization of submodular functions has received relatively little research attention. Our framework relies on an equivalent convex reformulation of a submodular minimization problem, which is efficiently computable. We then use this relaxation to exploit methods for the distributed optimization of convex functions. The proposed framework is applicable to submodular set functions as well as to a wider class of submodular functions defined over certain lattices. We also propose an approach for solving distributed motion coordination problems in discrete state space based on submodular function minimization. We establish through a challenging setup of capture the flag game that submodular functions over lattices can be used to design artificial potential fields over discrete state space in which the agents are attracted towards their goals and are repulsed from obstacles and from each other for collision avoidance. △ Less

Submitted 21 January, 2018; v1 submitted 21 September, 2017; originally announced September 2017.

An LP Approach for Solving Two-Player Zero-Sum Repeated Bayesian Games

Authors: Lichun Li, Cedric Langbort, Jeff Shamma

Abstract: This paper studies two-player zero-sum repeated Bayesian games in which every player has a private type that is unknown to the other player, and the initial probability of the type of every player is publicly known. The types of players are independently chosen according to the initial probabilities, and are kept the same all through the game. At every stage, players simultaneously choose actions,… ▽ More This paper studies two-player zero-sum repeated Bayesian games in which every player has a private type that is unknown to the other player, and the initial probability of the type of every player is publicly known. The types of players are independently chosen according to the initial probabilities, and are kept the same all through the game. At every stage, players simultaneously choose actions, and announce their actions publicly. For finite horizon cases, an explicit linear program is provided to compute players' security strategies. Moreover, based on the existing results in [1], this paper shows that a player's sufficient statistics, which is independent of the strategy of the other player, consists of the belief over the player's own type, the regret with respect to the other player's type, and the stage. Explicit linear programs are provided to compute the initial regrets, and the security strategies that only depends on the sufficient statistics. For discounted cases, following the same idea in the finite horizon, this paper shows that a player's sufficient statistics consists of the belief of the player's own type and the anti-discounted regret with respect to the other player's type. Besides, an approximated security strategy depending on the sufficient statistics is provided, and an explicit linear program to compute the approximated security strategy is given. This paper also obtains a bound on the performance difference between the approximated security strategy and the security strategy. △ Less

Submitted 7 November, 2017; v1 submitted 6 March, 2017; originally announced March 2017.

Comments: submitted to TAC, under review

Efficient Strategy Computation in Zero-Sum Asymmetric Repeated Games

Authors: Lichun Li, Jeff S. Shamma

Abstract: Zero-sum asymmetric games model decision making scenarios involving two competing players who have different information about the game being played. A particular case is that of nested information, where one (informed) player has superior information over the other (uninformed) player. This paper considers the case of nested information in repeated zero-sum games and studies the computation of st… ▽ More Zero-sum asymmetric games model decision making scenarios involving two competing players who have different information about the game being played. A particular case is that of nested information, where one (informed) player has superior information over the other (uninformed) player. This paper considers the case of nested information in repeated zero-sum games and studies the computation of strategies for both the informed and uninformed players for finite-horizon and discounted infinite-horizon nested information games. For finite-horizon settings, we exploit that for both players, the security strategy, and also the opponent's corresponding best response depend only on the informed player's history of actions. Using this property, we refine the sequence form, and formulate an LP computation of player strategies that is linear in the size of the uninformed player's action set. For the infinite-horizon discounted game, we construct LP formulations to compute the approximated security strategies for both players, and provide a bound on the performance difference between the approximated security strategies and the security strategies. Finally, we illustrate the results on a network interdiction game between an informed system administrator and uniformed intruder. △ Less

Submitted 7 November, 2017; v1 submitted 6 March, 2017; originally announced March 2017.

Comments: sumbitted to IEEE TAC, under review

Passivity Analysis of Higher Order Evolutionary Dynamics and Population Games

Authors: M. A. Mabrok, Jeff Shamma

Abstract: In population games, a large population of players, modeled as a continuum, is divided into subpopulations, and the fitness or payoff of each subpopulation depends on the overall population composition. Evolutionary dynamics describe how the population composition changes in response to the fitness levels, resulting in a closed-loop feedback system. Recent work established a connection between pas… ▽ More In population games, a large population of players, modeled as a continuum, is divided into subpopulations, and the fitness or payoff of each subpopulation depends on the overall population composition. Evolutionary dynamics describe how the population composition changes in response to the fitness levels, resulting in a closed-loop feedback system. Recent work established a connection between passivity theory and certain classes of population games, namely so-called "stable games". In particular, it was shown that a combination of stable games and (an analogue of) passive evolutionary dynamics results in stable convergence to Nash equilibrium. This paper considers the converse question of necessary conditions for evolutionary dynamics to exhibit stable behaviors for all generalized stable games. Here, generalization refers to "higher order" games where the population payoffs may be a dynamic function of the population state. Using methods from robust control analysis, we show that if an evolutionary dynamic does not satisfy a passivity property, then it is possible to construct a generalized stable game that results in instability. The results are illustrated on selected evolutionary dynamics with particular attention to replicator dynamics, which are also shown to be lossless, a special class of passive systems. △ Less

Submitted 16 September, 2016; originally announced September 2016.

Energy Aware Architecture for Coordinated Mobility: An Approximate Dynamic Programming Approach

Authors: Hassan Jaleel, Jeff S. Shamma

Abstract: Our goal is to design distributed coordination strategies that enable agents to achieve global performance guarantees while minimizing the energy cost of their actions with an emphasis on feasibility for real-time implementation. As a motivating scenario that illustrates the importance of introducing energy awareness at the agent level, we consider a team of mobile nodes that are assigned the task… ▽ More Our goal is to design distributed coordination strategies that enable agents to achieve global performance guarantees while minimizing the energy cost of their actions with an emphasis on feasibility for real-time implementation. As a motivating scenario that illustrates the importance of introducing energy awareness at the agent level, we consider a team of mobile nodes that are assigned the task of establishing a communication link between two base stations with minimum energy consumption. We formulate this problem as a dynamic program in which the total cost of each agent is the sum of both mobility and communication costs. To ensure that the solution is distributed and real time implementable, we propose multiple suboptimal policies based on the concepts of approximate dynamic programming. To provide performance guarantees, we compute upper bounds on the performance gap between the proposed suboptimal policies and the global optimal policy. Finally, we discuss merits and demerits of the proposed policies and compare their performance using simulations. △ Less

Submitted 25 July, 2016; originally announced July 2016.

Comments: 25 pages

Networked SIS Epidemics with Awareness

Authors: Keith Paarporn, Ceyhun Eksin, Joshua S. Weitz, Jeff S. Shamma

Abstract: We study an SIS epidemic process over a static contact network where the nodes have partial information about the epidemic state. They react by limiting their interactions with their neighbors when they believe the epidemic is currently prevalent. A node's awareness is weighted by the fraction of infected neighbors in their social network, and a global broadcast of the fraction of infected nodes i… ▽ More We study an SIS epidemic process over a static contact network where the nodes have partial information about the epidemic state. They react by limiting their interactions with their neighbors when they believe the epidemic is currently prevalent. A node's awareness is weighted by the fraction of infected neighbors in their social network, and a global broadcast of the fraction of infected nodes in the entire network. The dynamics of the benchmark (no awareness) and awareness models are described by discrete-time Markov chains, from which mean-field approximations (MFA) are derived. The states of the MFA are interpreted as the nodes' probabilities of being infected. We show a sufficient condition for existence of a "metastable", or endemic, state of the awareness model coincides with that of the benchmark model. Furthermore, we use a coupling technique to give a full stochastic comparison analysis between the two chains, which serves as a probabilistic analogue to the MFA analysis. In particular, we show that adding awareness reduces the expectation of any epidemic metric on the space of sample paths, e.g. eradication time or total infections. We characterize the reduction in expectations in terms of the coupling distribution. In simulations, we evaluate the effect social distancing has on contact networks from different random graph families (geometric, Erdős-Renyi, and scale-free random networks). △ Less

Submitted 12 July, 2016; v1 submitted 8 July, 2016; originally announced July 2016.

Comments: 10 pages, 5 figures

BLMA: A Blind Matching Algorithm with Application to Cognitive Radio Networks

Authors: Doha Hamza, Jeff S. Shamma

Abstract: We consider a two-sided matching problem with a defined notion of pairwise stability. We propose a distributed blind matching algorithm (BLMA) to solve the problem. We prove the solution produced by BLMA will converge to an $ε$-pairwise stable outcome with probability one. We then consider a matching problem in cognitive radio networks. Secondary users (SUs) are allowed access time to the spectrum… ▽ More We consider a two-sided matching problem with a defined notion of pairwise stability. We propose a distributed blind matching algorithm (BLMA) to solve the problem. We prove the solution produced by BLMA will converge to an $ε$-pairwise stable outcome with probability one. We then consider a matching problem in cognitive radio networks. Secondary users (SUs) are allowed access time to the spectrum belonging to the primary users (PUs) provided that they relay primary messages. We propose a realization of the BLMA to produce an $ε$-pairwise stable solution assuming quasi-convex and quasi-concave utilities. In the case of more general utility forms, we show another BLMA realization to provide a stable solution. Furthermore, we propose negotiation mechanism to bias the algorithm towards one side of the market. We use this mechanism to protect the exclusive rights of the PUs to the spectrum. In all such implementations of the BLMA, we impose a limited information exchange in the network so that agents can only calculate their own utilities, but no information is available about the utilities of any other users in the network. △ Less

Submitted 1 May, 2016; originally announced May 2016.

Disease dynamics on a network game: a little empathy goes a long way

Authors: Ceyhun Eksin, Jeff S. Shamma, Joshua S. Weitz

Abstract: Individuals change their behavior during an epidemic in response to whether they and/or those they interact with are healthy or sick. Healthy individuals are concerned about contracting a disease from their sick contacts and may utilize protective measures. Sick individuals may be concerned with spreading the disease to their healthy contacts and adopt preemptive measures. Yet, in practice both pr… ▽ More Individuals change their behavior during an epidemic in response to whether they and/or those they interact with are healthy or sick. Healthy individuals are concerned about contracting a disease from their sick contacts and may utilize protective measures. Sick individuals may be concerned with spreading the disease to their healthy contacts and adopt preemptive measures. Yet, in practice both protective and preemptive changes in behavior come with costs. This paper proposes a stochastic network disease game model that captures the self-interests of individuals during the spread of a susceptible-infected-susceptible (SIS) disease where individuals react to current risk of disease spread, and their reactions together with the current state of the disease stochastically determine the next stage of the disease. We show that there is a critical level of concern, i.e., empathy, by the sick individuals above which disease is eradicated fast. Furthermore, we find that if the network and disease parameters are above the epidemic threshold, the risk averse behavior by the healthy individuals cannot eradicate the disease without the preemptive measures of the sick individuals. This imbalance in the role played by the response of the infected versus the susceptible individuals in disease eradication affords critical policy insights. △ Less

Submitted 15 April, 2016; v1 submitted 12 April, 2016; originally announced April 2016.

Comments: 27 pages, 9 figures, submitted for publication

Learning Efficient Correlated Equilibria

Authors: Holly P. Borowski, Jason R. Marden, Jeff S. Shamma

Abstract: The majority of distributed learning literature focuses on convergence to Nash equilibria. Correlated equilibria, on the other hand, can often characterize more efficient collective behavior than even the best Nash equilibrium. However, there are no existing distributed learning algorithms that converge to specific correlated equilibria. In this paper, we provide one such algorithm which guarantee… ▽ More The majority of distributed learning literature focuses on convergence to Nash equilibria. Correlated equilibria, on the other hand, can often characterize more efficient collective behavior than even the best Nash equilibrium. However, there are no existing distributed learning algorithms that converge to specific correlated equilibria. In this paper, we provide one such algorithm which guarantees that the agents' collective joint strategy will constitute an efficient correlated equilibrium with high probability. The key to attaining efficient correlated behavior through distributed learning involves incorporating a common random signal into the learning environment. △ Less

Submitted 7 December, 2015; originally announced December 2015.

Comments: 11 pages, 1 figure

Global Games with Noisy Information Sharing

Authors: Hessam Mahdavifar, Ahmad Beirami, Behrouz Touri, Jeff S. Shamma

Abstract: Global games form a subclass of games with incomplete information where a set of agents decide actions against a regime with an underlying fundamental $θ$ representing its power. Each agent has access to an independent noisy observation of $θ$. In order to capture the behavior of agents in a social network of information exchange we assume that agents share their observation in a noisy environment… ▽ More Global games form a subclass of games with incomplete information where a set of agents decide actions against a regime with an underlying fundamental $θ$ representing its power. Each agent has access to an independent noisy observation of $θ$. In order to capture the behavior of agents in a social network of information exchange we assume that agents share their observation in a noisy environment prior to making their decision. We show that global games with noisy sharing of information do not admit an intuitive type of threshold policy which only depends on agents' belief about the underlying $θ$. This is in contrast to the existing results on the threshold policy for the conventional set-up of global games. Motivated by this result, we investigate the existence of equilibrium strategies in a more general collection of threshold-type policies and show that such equilibrium strategies exist and are unique if the sharing of information happens over a sufficiently noisy environment. △ Less

Submitted 28 October, 2017; v1 submitted 28 October, 2015; originally announced October 2015.

Comments: Accepted to IEEE Transactions on Signal and Information Processing over Networks

A Game-theoretic Formulation of the Homogeneous Self-Reconfiguration Problem

Authors: Daniel Pickem, Magnus Egerstedt, Jeff S. Shamma

Abstract: In this paper we formulate the homogeneous two- and three-dimensional self-reconfiguration problem over discrete grids as a constrained potential game. We develop a game-theoretic learning algorithm based on the Metropolis-Hastings algorithm that solves the self-reconfiguration problem in a globally optimal fashion. Both a centralized and a fully distributed algorithm are presented and we show tha… ▽ More In this paper we formulate the homogeneous two- and three-dimensional self-reconfiguration problem over discrete grids as a constrained potential game. We develop a game-theoretic learning algorithm based on the Metropolis-Hastings algorithm that solves the self-reconfiguration problem in a globally optimal fashion. Both a centralized and a fully distributed algorithm are presented and we show that the only stochastically stable state is the potential function maximizer, i.e. the desired target configuration. These algorithms compute transition probabilities in such a way that even though each agent acts in a self-interested way, the overall collective goal of self-reconfiguration is achieved. Simulation results confirm the feasibility of our approach and show convergence to desired target configurations. △ Less

Submitted 2 September, 2015; originally announced September 2015.

Comments: 8 pages, 5 figures, 2 algorithms

Communication-Free Distributed Coverage for Networked Systems

Authors: A. Yasin Yazicioglu, Magnus Egerstedt, Jeff S. Shamma

Abstract: In this paper, we present a communication-free algorithm for distributed coverage of an arbitrary network by a group of mobile agents with local sensing capabilities. The network is represented as a graph, and the agents are arbitrarily deployed on some nodes of the graph. Any node of the graph is covered if it is within the sensing range of at least one agent. The agents are mobile devices that a… ▽ More In this paper, we present a communication-free algorithm for distributed coverage of an arbitrary network by a group of mobile agents with local sensing capabilities. The network is represented as a graph, and the agents are arbitrarily deployed on some nodes of the graph. Any node of the graph is covered if it is within the sensing range of at least one agent. The agents are mobile devices that aim to explore the graph and to optimize their locations in a decentralized fashion by relying only on their sensory inputs. We formulate this problem in a game theoretic setting and propose a communication-free learning algorithm for maximizing the coverage. △ Less

Submitted 23 May, 2015; originally announced May 2015.

Formation of Robust Multi-Agent Networks Through Self-Organizing Random Regular Graphs

Authors: A. Yasin Yazicioglu, Magnus Egerstedt, Jeff S. Shamma

Abstract: Multi-agent networks are often modeled as interaction graphs, where the nodes represent the agents and the edges denote some direct interactions. The robustness of a multi-agent network to perturbations such as failures, noise, or malicious attacks largely depends on the corresponding graph. In many applications, networks are desired to have well-connected interaction graphs with relatively small… ▽ More Multi-agent networks are often modeled as interaction graphs, where the nodes represent the agents and the edges denote some direct interactions. The robustness of a multi-agent network to perturbations such as failures, noise, or malicious attacks largely depends on the corresponding graph. In many applications, networks are desired to have well-connected interaction graphs with relatively small number of links. One family of such graphs is the random regular graphs. In this paper, we present a decentralized scheme for transforming any connected interaction graph with a possibly non-integer average degree of k into a connected random m-regular graph for some m in [k, k + 2]. Accordingly, the agents improve the robustness of the network with a minimal change in the overall sparsity by optimizing the graph connectivity through the proposed local operations. △ Less

Submitted 27 March, 2015; originally announced March 2015.

Dynamics in atomic signaling games

Authors: Michael J. Fox, Behrouz Touri, Jeff S. Shamma

Abstract: We study an atomic signaling game under stochastic evolutionary dynamics. There is a finite number of players who repeatedly update from a finite number of available languages/signaling strategies. Players imitate the most fit agents with high probability or mutate with low probability. We analyze the long-run distribution of states and show that, for sufficiently small mutation probability, its s… ▽ More We study an atomic signaling game under stochastic evolutionary dynamics. There is a finite number of players who repeatedly update from a finite number of available languages/signaling strategies. Players imitate the most fit agents with high probability or mutate with low probability. We analyze the long-run distribution of states and show that, for sufficiently small mutation probability, its support is limited to efficient communication systems. We find that this behavior is insensitive to the particular choice of evolutionary dynamic, a property that is due to the game having a potential structure with a potential function corresponding to average fitness. Consequently, the model supports conclusions similar to those found in the literature on language competition. That is, we show that efficient languages eventually predominate the society while reproducing the empirical phenomenon of linguistic drift. The emergence of efficiency in the atomic case can be contrasted with results for non-atomic signaling games that establish the non-negligible possibility of convergence, under replicator dynamics, to states of unbounded efficiency loss. △ Less

Submitted 20 December, 2013; originally announced December 2013.

Gain Scheduling Control of Gas Turbine Engines: Absolute Stability by Finding a Common Lyapunov Matrix

Authors: Mehrdad Pakmehr, Nathan Fitzgerald, Eric Feron, Jeff Shamma, Alireza Behbahani

Abstract: This manuscript aims to develop and describe gain scheduling control concept for a gas turbine engine which drives a variable pitch propeller. An architecture for gain-scheduling control is developed that controls the turboshaft engine for large thrust commands in stable fashion with good performance. Fuel ow and propeller pitch angle are the two control inputs of the system. New stability proof h… ▽ More This manuscript aims to develop and describe gain scheduling control concept for a gas turbine engine which drives a variable pitch propeller. An architecture for gain-scheduling control is developed that controls the turboshaft engine for large thrust commands in stable fashion with good performance. Fuel ow and propeller pitch angle are the two control inputs of the system. New stability proof has been developed for gain scheduling control of gas turbine engines using global linearization and LMI techniques. This approach guarantees absolute stability of the closed loop gas turbine engine systems with gain-scheduling controllers. △ Less

Submitted 22 June, 2012; originally announced June 2012.

Comments: 15 pages, 21 figures

Aspiration Learning in Coordination Games

Authors: Georgios C. Chasparis, Ari Arapostathis, Jeff S. Shamma

Abstract: We consider the problem of distributed convergence to efficient outcomes in coordination games through dynamics based on aspiration learning. Under aspiration learning, a player continues to play an action as long as the rewards received exceed a specified aspiration level. Here, the aspiration level is a fading memory average of past rewards, and these levels also are subject to occasional random… ▽ More We consider the problem of distributed convergence to efficient outcomes in coordination games through dynamics based on aspiration learning. Under aspiration learning, a player continues to play an action as long as the rewards received exceed a specified aspiration level. Here, the aspiration level is a fading memory average of past rewards, and these levels also are subject to occasional random perturbations. A player becomes dissatisfied whenever a received reward is less than the aspiration level, in which case the player experiments with a probability proportional to the degree of dissatisfaction. Our first contribution is the characterization of the asymptotic behavior of the induced Markov chain of the iterated process in terms of an equivalent finite-state Markov chain. We then characterize explicitly the behavior of the proposed aspiration learning in a generalized version of coordination games, examples of which include network formation and common-pool games. In particular, we show that in generic coordination games the frequency at which an efficient action profile is played can be made arbitrarily large. Although convergence to efficient outcomes is desirable, in several coordination games, such as common-pool games, attainability of fair outcomes, i.e., sequences of plays at which players experience highly rewarding returns with the same frequency, might also be of special interest. To this end, we demonstrate through analysis and simulations that aspiration learning also establishes fair outcomes in all symmetric coordination games, including common-pool games. △ Less

Submitted 19 October, 2011; originally announced October 2011.

Comments: 27 pages

MSC Class: 68T05; 91A26; 91A22; 93E35; 60J05; 91A80

Journal ref: SIAM J. Control Optim. 51 (2013), no. 1, 465-490

Multiple-Model Adaptive Control With Set-Valued Observers

Authors: Paulo Rosa, Carlos Silvestre, Jeff S. Shamma, Michael Athans

Abstract: This paper proposes a multiple-model adaptive control methodology, using set-valued observers (MMAC-SVO) for the identification subsystem, that is able to provide robust stability and performance guarantees for the closed-loop, when the plant, which can be open-loop stable or unstable, has significant parametric uncertainty. We illustrate, with an example, how set-valued observers (SVOs) can be… ▽ More This paper proposes a multiple-model adaptive control methodology, using set-valued observers (MMAC-SVO) for the identification subsystem, that is able to provide robust stability and performance guarantees for the closed-loop, when the plant, which can be open-loop stable or unstable, has significant parametric uncertainty. We illustrate, with an example, how set-valued observers (SVOs) can be used to select regions of uncertainty for the parameters of the plant. We also discuss some of the most problematic computational shortcomings and numerical issues that arise from the use of this kind of robust estimation methods. The behavior of the proposed control algorithm is demonstrated in simulation. △ Less

Submitted 2 October, 2009; v1 submitted 7 September, 2009; originally announced September 2009.

Comments: Combined 48th IEEE Conference on Decision and Control and 28th Chinese Control Conference, 2009