Showing 1–33 of 33 results for author: Pavel, L

Robust Accelerated Dynamics for Subnetwork Bilinear Zero-Sum Games with Distributed Restarting

Authors: Weijian Li, Lacra Pavel, and Andreas A. Malikopoulos

Abstract: In this paper, we investigate distributed Nash equilibrium seeking for a class of two-subnetwork zero-sum games characterized by bilinear coupling. We present a distributed primal-dual accelerated mirror-descent algorithm that guarantees convergence. However, we demonstrate that this time-varying algorithm is not robust, as it fails to converge under even the slightest disturbances. To address thi… ▽ More In this paper, we investigate distributed Nash equilibrium seeking for a class of two-subnetwork zero-sum games characterized by bilinear coupling. We present a distributed primal-dual accelerated mirror-descent algorithm that guarantees convergence. However, we demonstrate that this time-varying algorithm is not robust, as it fails to converge under even the slightest disturbances. To address this limitation, we introduce a distributed accelerated algorithm that employs a coordinated restarting mechanism. We model this new algorithm as a hybrid dynamical system and establish that it possesses structural robustness. △ Less

Submitted 23 April, 2025; originally announced April 2025.

Primal-dual Accelerated Mirror-Descent Method for Constrained Bilinear Saddle-Point Problems

Authors: Weijian Li, Xianlin Zeng, Lacra Pavel

Abstract: We develop a first-order accelerated algorithm for a class of constrained bilinear saddle-point problems with applications to network systems. The algorithm is a modified time-varying primal-dual version of an accelerated mirror-descent dynamics. It deals with constraints such as simplices and convex set constraints effectively, and converges with a rate of $O(1/t^2)$. Furthermore, we employ the a… ▽ More We develop a first-order accelerated algorithm for a class of constrained bilinear saddle-point problems with applications to network systems. The algorithm is a modified time-varying primal-dual version of an accelerated mirror-descent dynamics. It deals with constraints such as simplices and convex set constraints effectively, and converges with a rate of $O(1/t^2)$. Furthermore, we employ the acceleration scheme to constrained distributed optimization and bilinear zero-sum games, and obtain two variants of distributed accelerated algorithms. △ Less

Submitted 3 October, 2024; v1 submitted 26 September, 2024; originally announced September 2024.

Passivity-based Gradient-Play Dynamics for Distributed Generalized Nash Equilibrium Seeking

Authors: Weijian Li, Lacra Pavel

Abstract: We consider seeking generalized Nash equilibria (GNE) for noncooperative games with coupled nonlinear constraints over networks. We first revisit a well-known gradientplay dynamics from a passivity-based perspective, and address that the strict monotonicity on pseudo-gradients is a critical assumption to ensure the exact convergence of the dynamics. Then we propose two novel passivity-based gradie… ▽ More We consider seeking generalized Nash equilibria (GNE) for noncooperative games with coupled nonlinear constraints over networks. We first revisit a well-known gradientplay dynamics from a passivity-based perspective, and address that the strict monotonicity on pseudo-gradients is a critical assumption to ensure the exact convergence of the dynamics. Then we propose two novel passivity-based gradient-play dynamics by introducing parallel feedforward compensators (PFCs) and output feedback compensators (OFCs). We show that the proposed dynamics can reach exact GNEs in merely monotone regimes if the PFCs are strictly passive or the OFCs are output strictly passive. Following that, resorting to passivity, we develop a unifying framework to generalize the gradient-play dynamics, and moreover, design a class of explicit passive-based dynamics with convergence guarantees. In addition, we explore the relation between the proposed dynamics and some existing methods, and extend our results to partial-decision information settings. △ Less

Submitted 22 August, 2024; originally announced August 2024.

Generalizing Better Response Paths and Weakly Acyclic Games

Authors: Bora Yongacoglu, Gürdal Arslan, Lacra Pavel, Serdar Yüksel

Abstract: Weakly acyclic games generalize potential games and are fundamental to the study of game theoretic control. In this paper, we present a generalization of weakly acyclic games, and we observe its importance in multi-agent learning when agents employ experimental strategy updates in periods where they fail to best respond. While weak acyclicity is defined in terms of path connectivity properties of… ▽ More Weakly acyclic games generalize potential games and are fundamental to the study of game theoretic control. In this paper, we present a generalization of weakly acyclic games, and we observe its importance in multi-agent learning when agents employ experimental strategy updates in periods where they fail to best respond. While weak acyclicity is defined in terms of path connectivity properties of a game's better response graph, our generalization is defined using a generalized better response graph. We provide sufficient conditions for this notion of generalized weak acyclicity in both two-player games and $n$-player games. To demonstrate that our generalization is not trivial, we provide examples of games admitting a pure Nash equilibrium that are not generalized weakly acyclic. The generalization presented in this work is closely related to the recent theory of satisficing paths, and the counterexamples presented here constitute the first negative results in that theory. △ Less

Submitted 26 March, 2024; originally announced March 2024.

Paths to Equilibrium in Games

Authors: Bora Yongacoglu, Gürdal Arslan, Lacra Pavel, Serdar Yüksel

Abstract: In multi-agent reinforcement learning (MARL) and game theory, agents repeatedly interact and revise their strategies as new data arrives, producing a sequence of strategy profiles. This paper studies sequences of strategies satisfying a pairwise constraint inspired by policy updating in reinforcement learning, where an agent who is best responding in one period does not switch its strategy in the… ▽ More In multi-agent reinforcement learning (MARL) and game theory, agents repeatedly interact and revise their strategies as new data arrives, producing a sequence of strategy profiles. This paper studies sequences of strategies satisfying a pairwise constraint inspired by policy updating in reinforcement learning, where an agent who is best responding in one period does not switch its strategy in the next period. This constraint merely requires that optimizing agents do not switch strategies, but does not constrain the non-optimizing agents in any way, and thus allows for exploration. Sequences with this property are called satisficing paths, and arise naturally in many MARL algorithms. A fundamental question about strategic dynamics is such: for a given game and initial strategy profile, is it always possible to construct a satisficing path that terminates at an equilibrium? The resolution of this question has implications about the capabilities or limitations of a class of MARL algorithms. We answer this question in the affirmative for normal-form games. Our analysis reveals a counterintuitive insight that reward deteriorating strategic updates are key to driving play to equilibrium along a satisficing path. △ Less

Submitted 1 October, 2024; v1 submitted 26 March, 2024; originally announced March 2024.

Comments: Accepted to NeurIPS 2024

Recursive Reasoning in Minimax Games: A Level $k$ Gradient Play Method

Authors: Zichu Liu, Lacra Pavel

Abstract: Despite the success of generative adversarial networks (GANs) in generating visually appealing images, they are notoriously challenging to train. In order to stabilize the learning dynamics in minimax games, we propose a novel recursive reasoning algorithm: Level $k$ Gradient Play (Lv.$k$ GP) algorithm. In contrast to many existing algorithms, our algorithm does not require sophisticated heuristic… ▽ More Despite the success of generative adversarial networks (GANs) in generating visually appealing images, they are notoriously challenging to train. In order to stabilize the learning dynamics in minimax games, we propose a novel recursive reasoning algorithm: Level $k$ Gradient Play (Lv.$k$ GP) algorithm. In contrast to many existing algorithms, our algorithm does not require sophisticated heuristics or curvature information. We show that as $k$ increases, Lv.$k$ GP converges asymptotically towards an accurate estimation of players' future strategy. Moreover, we justify that Lv.$\infty$ GP naturally generalizes a line of provably convergent game dynamics which rely on predictive updates. Furthermore, we provide its local convergence property in nonconvex-nonconcave zero-sum games and global convergence in bilinear and quadratic games. By combining Lv.$k$ GP with Adam optimizer, our algorithm shows a clear advantage in terms of performance and computational overhead compared to other methods. Using a single Nvidia RTX3090 GPU and 30 times fewer parameters than BigGAN on CIFAR-10, we achieve an FID of 10.17 for unconditional image generation within 30 hours, allowing GAN training on common computational resources to reach state-of-the-art performance. △ Less

Submitted 28 October, 2022; originally announced October 2022.

Comments: For the code associated with this paper, see https://github.com/ZichuLiu/submission

Resilient Nash Equilibrium Seeking in the Partial Information Setting

Authors: Dian Gadjov, Lacra Pavel

Abstract: Current research in distributed Nash equilibrium (NE) seeking in the partial information setting assumes that information is exchanged between agents that are "truthful". However, in general noncooperative games agents may consider sending misinformation to neighboring agents with the goal of further reducing their cost. Additionally, communication networks are vulnerable to attacks from agents ou… ▽ More Current research in distributed Nash equilibrium (NE) seeking in the partial information setting assumes that information is exchanged between agents that are "truthful". However, in general noncooperative games agents may consider sending misinformation to neighboring agents with the goal of further reducing their cost. Additionally, communication networks are vulnerable to attacks from agents outside the game as well as communication failures. In this paper, we propose a distributed NE seeking algorithm that is robust against adversarial agents that transmit noise, random signals, constant singles, deceitful messages, as well as being resilient to external factors such as dropped communication, jammed signals, and man in the middle attacks. The core issue that makes the problem challenging is that agents have no means of verifying if the information they receive is correct, i.e. there is no "ground truth". To address this problem, we use an observation graph, that gives truthful action information, in conjunction with a communication graph, that gives (potentially incorrect) information. By filtering information obtained from these two graphs, we show that our algorithm is resilient against adversarial agents and converges to the Nash equilibrium. △ Less

Submitted 26 November, 2021; originally announced November 2021.

Second-Order Mirror Descent: Convergence in Games Beyond Averaging and Discounting

Authors: Bolin Gao, Lacra Pavel

Abstract: In this paper, we propose a second-order extension of the continuous-time game-theoretic mirror descent (MD) dynamics, referred to as MD2, which provably converges to mere (but not necessarily strict) variationally stable states (VSS) without using common auxiliary techniques such as time-averaging or discounting. We show that MD2 enjoys no-regret as well as an exponential rate of convergence towa… ▽ More In this paper, we propose a second-order extension of the continuous-time game-theoretic mirror descent (MD) dynamics, referred to as MD2, which provably converges to mere (but not necessarily strict) variationally stable states (VSS) without using common auxiliary techniques such as time-averaging or discounting. We show that MD2 enjoys no-regret as well as an exponential rate of convergence towards strong VSS upon a slight modification. MD2 can also be used to derive many novel continuous-time primal-space dynamics. We then use stochastic approximation techniques to provide a convergence guarantee of discrete-time MD2 with noisy observations towards interior mere VSS. Selected simulations are provided to illustrate our results. △ Less

Submitted 30 June, 2023; v1 submitted 18 November, 2021; originally announced November 2021.

Comments: 16 pages, 12 figures. This work has been submitted to the IEEE for possible publication

An inexact-penalty method for GNE seeking in games with dynamic agents

Authors: Andrew R. Romano, Lacra Pavel

Abstract: We consider a network of autonomous agents whose outputs are actions in a game with coupled constraints. In such network scenarios, agents seeking to minimize coupled cost functions using distributed information while satisfying the coupled constraints. Current methods consider the small class of multi-integrator agents using primal-dual methods. These methods can only ensure constraint satisfacti… ▽ More We consider a network of autonomous agents whose outputs are actions in a game with coupled constraints. In such network scenarios, agents seeking to minimize coupled cost functions using distributed information while satisfying the coupled constraints. Current methods consider the small class of multi-integrator agents using primal-dual methods. These methods can only ensure constraint satisfaction in steady-state. In contrast, we propose an inexact penalty method using a barrier function for nonlinear agents with equilibrium-independent passive dynamics. We show that these dynamics converge to an epsilon-GNE while satisfying the constraints for all time, not only in steady-state. We develop these dynamics in both the full-information and partial-information settings. In the partial-information setting, dynamic estimates of the others' actions are used to make decisions and are updated through local communication. Applications to optical networks and velocity synchronization of flexible robots are provided. △ Less

Submitted 23 April, 2021; originally announced April 2021.

Comments: 10 pages, 5 figures

On the exact convergence to Nash equilibrium in hypomonotone regimes under full and partial-information

Authors: Dian Gadjov, Lacra Pavel

Abstract: In this paper, we consider distributed Nash equilibrium seeking in monotone and hypomonotone games. We first assume that each player has knowledge of the opponents' decisions and propose a passivity-based modification of the standard gradient-play dynamics, that we call "Heavy Anchor". We prove that Heavy Anchor allows a relaxation of strict monotonicity of the pseudo-gradient, needed for gradient… ▽ More In this paper, we consider distributed Nash equilibrium seeking in monotone and hypomonotone games. We first assume that each player has knowledge of the opponents' decisions and propose a passivity-based modification of the standard gradient-play dynamics, that we call "Heavy Anchor". We prove that Heavy Anchor allows a relaxation of strict monotonicity of the pseudo-gradient, needed for gradient-play dynamics, and can ensure exact asymptotic convergence in merely monotone regimes. We extend these results to the setting where each player has only partial information of the opponents' decisions. Each player maintains a local decision variable and an auxiliary state estimate and communicates with their neighbours to learn the opponents' actions. We modify Heavy Anchor via a distributed Laplacian feedback and show how we can exploit equilibrium-independent passivity properties to achieve convergence to a Nash equilibrium in hypomonotone regimes. △ Less

Submitted 22 April, 2021; originally announced April 2021.

Continuous-Time Convergence Rates in Potential and Monotone Games

Authors: Bolin Gao, Lacra Pavel

Abstract: In this paper, we provide exponential rates of convergence to the interior Nash equilibrium for continuous-time dual-space game dynamics such as mirror descent (MD) and actor-critic (AC). We perform our analysis in $N$-player continuous concave games that satisfy certain monotonicity assumptions while possibly also admitting potential functions. In the first part of this paper, we provide a novel… ▽ More In this paper, we provide exponential rates of convergence to the interior Nash equilibrium for continuous-time dual-space game dynamics such as mirror descent (MD) and actor-critic (AC). We perform our analysis in $N$-player continuous concave games that satisfy certain monotonicity assumptions while possibly also admitting potential functions. In the first part of this paper, we provide a novel relative characterization of monotone games and show that MD and its discounted version converge with $\mathcal{O}(e^{-βt})$ in relatively strongly and relatively hypo-monotone games, respectively. In the second part of this paper, we specialize our results to games that admit a relatively strongly concave potential and show AC converges with $\mathcal{O}(e^{-βt})$. These rates extend their known convergence conditions. Simulations are performed which empirically back up our results. △ Less

Submitted 2 February, 2022; v1 submitted 20 November, 2020; originally announced November 2020.

Comments: 20 pages, 5 figures, manuscript submitted to SIAM Journal on Control and Optimization (SICON) for possible publication

Continuous-time Discounted Mirror-Descent Dynamics in Monotone Concave Games

Authors: Bolin Gao, Lacra Pavel

Abstract: In this paper, we consider concave continuous-kernel games characterized by monotonicity properties and propose discounted mirror descent-type dynamics. We introduce two classes of dynamics whereby the associated mirror map is constructed based on a strongly convex or a Legendre regularizer. Depending on the properties of the regularizer we show that these new dynamics can converge asymptotically… ▽ More In this paper, we consider concave continuous-kernel games characterized by monotonicity properties and propose discounted mirror descent-type dynamics. We introduce two classes of dynamics whereby the associated mirror map is constructed based on a strongly convex or a Legendre regularizer. Depending on the properties of the regularizer we show that these new dynamics can converge asymptotically in concave games with monotone (negative) pseudo-gradient. Furthermore, we show that when the regularizer enjoys strong convexity, the resulting dynamics can converge even in games with hypo-monotone (negative) pseudo-gradient, which corresponds to a shortage of monotonicity. △ Less

Submitted 7 December, 2019; originally announced December 2019.

Comments: 8 pages, 9 figures. This work has been submitted to the IEEE for possible publication

Journal ref: IEEE Transactions on Automatic Control, vol 66 (11), 2021

Single-timescale distributed GNE seeking for aggregative games over networks via forward-backward operator splitting

Authors: Dian Gadjov, Lacra Pavel

Abstract: We consider aggregative games with affine coupling constraints, where agents have partial information on the aggregate value and can only communicate with neighbouring agents. We propose a single-layer distributed algorithm that reaches a variational generalized Nash equilibrium, under constant step sizes. The algorithm works on a single timescale, i.e., does not require multiple communication rou… ▽ More We consider aggregative games with affine coupling constraints, where agents have partial information on the aggregate value and can only communicate with neighbouring agents. We propose a single-layer distributed algorithm that reaches a variational generalized Nash equilibrium, under constant step sizes. The algorithm works on a single timescale, i.e., does not require multiple communication rounds between agents before updating their action. The convergence proof leverages an invariance property of the aggregate estimates and relies on a forward-backward splitting for two preconditioned operators and their restricted (strong) monotonicity properties on the consensus subspace. △ Less

Submitted 4 August, 2020; v1 submitted 31 July, 2019; originally announced August 2019.

Comments: 8 pages, 8 figures, submitted to TAC

Journal ref: IEEE Transactions on Automatic Control, 2020

Dynamic NE Seeking for Multi-Integrator Networked Agents with Disturbance Rejection

Authors: Andrew R Romano, Lacra Pavel

Abstract: In this paper, we consider game problems played by (multi)-integrator agents, subject to external disturbances. We propose Nash equilibrium seeking dynamics based on gradient-play, augmented with a dynamic internal-model based component, which is a reduced-order observer of the disturbance. We consider single-, double- and extensions to multi-integrator agents, in a partial-information setting, wh… ▽ More In this paper, we consider game problems played by (multi)-integrator agents, subject to external disturbances. We propose Nash equilibrium seeking dynamics based on gradient-play, augmented with a dynamic internal-model based component, which is a reduced-order observer of the disturbance. We consider single-, double- and extensions to multi-integrator agents, in a partial-information setting, where agents have only partial knowledge on the others' decisions over a network. The lack of global information is offset by each agent maintaining an estimate of the others' states, based on local communication with its neighbours. Each agent has an additional dynamic component that drives its estimates to the consensus subspace. In all cases, we show convergence to the Nash equilibrium irrespective of disturbances. Our proofs leverage input-to-state stability under strong monotonicity of the pseudo-gradient and Lipschitz continuity of the extended pseudo-gradient. △ Less

Submitted 6 March, 2019; originally announced March 2019.

Journal ref: IEEE Transactions on Control of Network Systems, vol. 7 (1), pp. 129-139, 2020

Distributed GNE seeking under partial-decision information over networks via a doubly-augmented operator splitting approach

Authors: Lacra Pavel

Abstract: We consider distributed computation of generalized Nash equilibrium (GNE) over networks, in games with shared coupling constraints. Existing methods require that each player has full access to opponents' decisions. In this paper, we assume that players have only partial-decision information, and can communicate with their neighbours over an arbitrary undirected graph. We recast the problem as that… ▽ More We consider distributed computation of generalized Nash equilibrium (GNE) over networks, in games with shared coupling constraints. Existing methods require that each player has full access to opponents' decisions. In this paper, we assume that players have only partial-decision information, and can communicate with their neighbours over an arbitrary undirected graph. We recast the problem as that of finding a zero of a sum of monotone operators through primal-dual analysis. To distribute the problem, we doubly augment variables, so that each player has local decision estimates and local copies of Lagrangian multipliers. We introduce a single-layer algorithm, fully distributed with respect to both primal and dual variables. We show its convergence to a variational GNE with fixed step-sizes, by reformulating it as a forward-backward iteration for a pair of doubly-augmented monotone operators. △ Less

Submitted 13 August, 2018; originally announced August 2018.

Comments: 12 pages, 5 figures. This work has been submitted to the IEEE for possible publication

Journal ref: IEEE Transactions on Automatic Control, vol. 65 (4), pp. 1584-1597, 2020

On Passivity, Reinforcement Learning and Higher-Order Learning in Multi-Agent Finite Games

Authors: Bolin Gao, Lacra Pavel

Abstract: In this paper, we propose a passivity-based methodology for analysis and design of reinforcement learning in multi-agent finite games. Starting from a known exponentially-discounted reinforcement learning scheme, we show that convergence to a Nash distribution can be shown in the class of games characterized by the monotonicity property of their (negative) payoff. We further exploit passivity to p… ▽ More In this paper, we propose a passivity-based methodology for analysis and design of reinforcement learning in multi-agent finite games. Starting from a known exponentially-discounted reinforcement learning scheme, we show that convergence to a Nash distribution can be shown in the class of games characterized by the monotonicity property of their (negative) payoff. We further exploit passivity to propose a class of higher-order schemes that preserve convergence properties, can improve the speed of convergence and can even converge in cases whereby their first-order counterpart fail to converge. We demonstrate these properties through numerical simulations for several representative games. △ Less

Submitted 13 August, 2018; originally announced August 2018.

Comments: 14 pages, 19 figures. This work has been submitted to the IEEE for possible publication

Journal ref: IEEE Transactions on Automatic Control, 2020

On seeking efficient Pareto optimal points in multi-player minimum cost flow problems with application to transportation systems

Authors: Shuvomoy Das Gupta, Lacra Pavel

Abstract: In this paper, we propose a multi-player extension of the minimum cost flow problem inspired by a transportation problem that arises in modern transportation industry. We associate one player with each arc of a directed network, each trying to minimize its cost function subject to the network flow constraints. In our model, the cost function can be any general nonlinear function, and the flow thro… ▽ More In this paper, we propose a multi-player extension of the minimum cost flow problem inspired by a transportation problem that arises in modern transportation industry. We associate one player with each arc of a directed network, each trying to minimize its cost function subject to the network flow constraints. In our model, the cost function can be any general nonlinear function, and the flow through each arc is an integer. We present algorithms to compute efficient Pareto optimal point(s), where the maximum possible number of players (but not all) minimize their cost functions simultaneously. The computed Pareto optimal points are Nash equilibriums if the problem is transformed into a finite static game in normal form. △ Less

Submitted 21 May, 2019; v1 submitted 29 May, 2018; originally announced May 2018.

Comments: 27 pages

Journal ref: Journal of Global Optimization, May, 2019

From Game-theoretic Multi-agent Log Linear Learning to Reinforcement Learning

Authors: Mohammadhosein Hasanbeig, Lacra Pavel

Abstract: The main focus of this paper is on enhancement of two types of game-theoretic learning algorithms: log-linear learning and reinforcement learning. The standard analysis of log-linear learning needs a highly structured environment, i.e. strong assumptions about the game from an implementation perspective. In this paper, we introduce a variant of log-linear learning that provides asymptotic guarante… ▽ More The main focus of this paper is on enhancement of two types of game-theoretic learning algorithms: log-linear learning and reinforcement learning. The standard analysis of log-linear learning needs a highly structured environment, i.e. strong assumptions about the game from an implementation perspective. In this paper, we introduce a variant of log-linear learning that provides asymptotic guarantees while relaxing the structural assumptions to include synchronous updates and limitations in information available to the players. On the other hand, model-free reinforcement learning is able to perform even under weaker assumptions on players' knowledge about the environment and other players' strategies. We propose a reinforcement algorithm that uses a double-aggregation scheme in order to deepen players' insight about the environment and constant learning step-size which achieves a higher convergence rate. Numerical experiments are conducted to verify each algorithm's robustness and performance. △ Less

Submitted 18 September, 2018; v1 submitted 6 February, 2018; originally announced February 2018.

Asynchronous distributed algorithms for seeking generalized Nash equilibria under full and partial decision information

Authors: Peng Yi, Lacra Pavel

Abstract: We investigate asynchronous distributed algorithms with delayed information for seeking generalized Nash equilibrium over multi-agent networks. The considered game model has all players' local decisions coupled with a shared affine constraint. We assume each player can only access its local objective function, local constraint, and a local block matrix of the affine constraint. We first give the a… ▽ More We investigate asynchronous distributed algorithms with delayed information for seeking generalized Nash equilibrium over multi-agent networks. The considered game model has all players' local decisions coupled with a shared affine constraint. We assume each player can only access its local objective function, local constraint, and a local block matrix of the affine constraint. We first give the algorithm for the case when each agent is able to fully access all other players' decisions. With the help of auxiliary edge variables and edge Laplacian matrix, each player can carry on its local iteration in an asynchronous manner, using only local data and possibly delayed neighbour information. And then we investigate the case when the agents cannot know all other players' decisions, which is called a partial decision case. We introduce a local estimation of the overall decisions for each agent in the partial decision case, and develop another asynchronous algorithm by incorporating consensus dynamics on the local estimations of the overall decisions. Since both algorithms do not need any centralized clock coordination, the algorithms fully exploit the local computation resource of each player, and remove the idle time due to waiting for the "slowest" agent. Both algorithms are developed by a preconditioned forward-backward operator splitting method, while convergence is shown with the help of asynchronous fixed-point iterations under proper assumptions and fixed step-size choices. Numerical studies verify both algorithms' convergence and efficiency. △ Less

Submitted 24 July, 2018; v1 submitted 9 January, 2018; originally announced January 2018.

Comments: 14 pages, 8 figures

Distributed Nash Equilibrium Seeking under Partial-Decision Information via the Alternating Direction Method of Multipliers

Authors: Farzad Salehisadaghiani, Wei Shi, Lacra Pavel

Abstract: In this paper, we consider the problem of finding a Nash equilibrium in a multi-player game over generally connected networks. This model differs from a conventional setting in that players have partial information on the actions of their opponents and the communication graph is not necessarily the same as the players' cost dependency graph. We develop a relatively fast algorithm within the framew… ▽ More In this paper, we consider the problem of finding a Nash equilibrium in a multi-player game over generally connected networks. This model differs from a conventional setting in that players have partial information on the actions of their opponents and the communication graph is not necessarily the same as the players' cost dependency graph. We develop a relatively fast algorithm within the framework of inexact-ADMM, based on local information exchange between the players. We prove its convergence to Nash equilibrium for fixed step-sizes and analyze its convergence rate. Numerical simulations illustrate its benefits when compared to a consensus-based gradient type algorithm with diminishing step-sizes. △ Less

Submitted 9 February, 2019; v1 submitted 6 July, 2017; originally announced July 2017.

Journal ref: Automatica, vol 103, pp. 27 - 35, 2019

A Passivity-Based Approach to Nash Equilibrium Seeking over Networks

Authors: Dian Gadjov, Lacra Pavel

Abstract: In this paper we consider the problem of distributed Nash equilibrium (NE) seeking over networks, a setting in which players have limited local information. We start from a continuous-time gradient-play dynamics that converges to an NE under strict monotonicity of the pseudo-gradient and assumes perfect information, i.e., instantaneous all-to-all player communication. We consider how to modify thi… ▽ More In this paper we consider the problem of distributed Nash equilibrium (NE) seeking over networks, a setting in which players have limited local information. We start from a continuous-time gradient-play dynamics that converges to an NE under strict monotonicity of the pseudo-gradient and assumes perfect information, i.e., instantaneous all-to-all player communication. We consider how to modify this gradient-play dynamics in the case of partial, or networked information between players. We propose an augmented gradient-play dynamics with correction in which players communicate locally only with their neighbours to compute an estimate of the other players' actions. We derive the new dynamics based on the reformulation as a multi-agent coordination problem over an undirected graph. We exploit incremental passivity properties and show that a synchronizing, distributed Laplacian feedback can be designed using relative estimates of the neighbours. Under a strict monotonicity property of the pseudo-gradient, we show that the augmented gradient-play dynamics converges to consensus on the NE of the game. We further discuss two cases that highlight the tradeoff between properties of the game and the communication graph. △ Less

Submitted 5 May, 2017; originally announced May 2017.

Comments: This work has been submitted to the IEEE for possible publication

Journal ref: IEEE Transactions on Automatic Control, vol 64, no 3, pp. 1077-1092, 2019

Distributed generalized Nash equilibria computation of monotone games via a preconditioned proximal point algorithm

Authors: Peng Yi, Lacra Pavel

Abstract: In this paper, we investigate distributed generalized Nash equilibrium (GNE) computation of monotone games with affine coupling constraints. Each player can only utilize its local objective function, local feasible set and a local block of the coupling constraint, and can only communicate with its neighbours. We assume the game has monotone pseudo-subdifferential without Lipschitz continuity restr… ▽ More In this paper, we investigate distributed generalized Nash equilibrium (GNE) computation of monotone games with affine coupling constraints. Each player can only utilize its local objective function, local feasible set and a local block of the coupling constraint, and can only communicate with its neighbours. We assume the game has monotone pseudo-subdifferential without Lipschitz continuity restrictions. We design novel center-free distributed GNE seeking algorithms for equality and inequality affine coupling constraints, respectively. A proximal alternating direction method of multipliers(ADMM) is proposed for the equality case, while for the inequality case, a parallel splitting type algorithm is proposed. In both algorithms, the GNE seeking task is decomposed into a sequential NE computation of regularized subgames and distributed update of multipliers and auxiliary variables, based on local data and local communication. Our two double-layer GNE algorithms need not specify the inner-loop NE seeking algorithm and moreover, only require that the strongly monotone subgames are inexactly solved. We prove their convergence by showing that the two algorithms can be seen as specific instances of preconditioned proximal point algorithms} (PPPA) for finding zeros of monotone operators. Applications and numerical simulations are given for illustration. △ Less

Submitted 30 September, 2017; v1 submitted 3 May, 2017; originally announced May 2017.

Comments: 13 pages, 7 figures

Journal ref: IEEE Transactions on Control of Network Systems, vol 6, no 1, pp. 299-311, 2019

On the Properties of the Softmax Function with Application in Game Theory and Reinforcement Learning

Authors: Bolin Gao, Lacra Pavel

Abstract: In this paper, we utilize results from convex analysis and monotone operator theory to derive additional properties of the softmax function that have not yet been covered in the existing literature. In particular, we show that the softmax function is the monotone gradient map of the log-sum-exp function. By exploiting this connection, we show that the inverse temperature parameter determines the L… ▽ More In this paper, we utilize results from convex analysis and monotone operator theory to derive additional properties of the softmax function that have not yet been covered in the existing literature. In particular, we show that the softmax function is the monotone gradient map of the log-sum-exp function. By exploiting this connection, we show that the inverse temperature parameter determines the Lipschitz and co-coercivity properties of the softmax function. We then demonstrate the usefulness of these properties through an application in game-theoretic reinforcement learning. △ Less

Submitted 20 August, 2018; v1 submitted 3 April, 2017; originally announced April 2017.

Comments: 10 pages, 4 figures. Comments are welcome

A Distributed Nash Equilibrium Seeking in Networked Graphical Games

Authors: Farzad Salehisadaghiani, Lacra Pavel

Abstract: This paper considers a distributed gossip approach for finding a Nash equilibrium in networked games on graphs. In such games a player's cost function may be affected by the actions of any subset of players. An interference graph is employed to illustrate the partially-coupled cost functions and the asymmetric information requirements. For a given interference graph, network communication between… ▽ More This paper considers a distributed gossip approach for finding a Nash equilibrium in networked games on graphs. In such games a player's cost function may be affected by the actions of any subset of players. An interference graph is employed to illustrate the partially-coupled cost functions and the asymmetric information requirements. For a given interference graph, network communication between players is considered to be limited. A generalized communication graph is designed so that players exchange only their required information. An algorithm is designed whereby players, with possibly partially-coupled cost functions, make decisions based on the estimates of other players' actions obtained from local neighbors. It is shown that this choice of communication graph guarantees that all players' information is exchanged after sufficiently many iterations. Using a set of standard assumptions on the cost functions, the interference and the communication graphs, almost sure convergence to a Nash equilibrium is proved for diminishing step sizes. Moreover, the case when the cost functions are not known by the players is investigated and a convergence proof is presented for diminishing step sizes. The effect of the second largest eigenvalue of the expected communication matrix on the convergence rate is quantified. The trade-off between parameters associated with the communication graph and the ones associated with the interference graph is illustrated. Numerical results are presented for a large-scale networked game. △ Less

Submitted 28 March, 2017; originally announced March 2017.

Journal ref: Automatica, 87, pp. 17 - 24, 2018

Generalized Nash Equilibrium Problem by the Alternating Direction Method of Multipliers

Authors: Farzad Salehisadaghiani, Lacra Pavel

Abstract: In this paper, the problem of finding a generalized Nash equilibrium (GNE) of a networked game is studied. Players are only able to choose their decisions from a feasible action set. The feasible set is considered to be a private linear equality constraint that is coupled through decisions of the other players. We consider that each player has his own private constraint and it has not to be shared… ▽ More In this paper, the problem of finding a generalized Nash equilibrium (GNE) of a networked game is studied. Players are only able to choose their decisions from a feasible action set. The feasible set is considered to be a private linear equality constraint that is coupled through decisions of the other players. We consider that each player has his own private constraint and it has not to be shared with the other players. This general case also embodies the one with shared constraints between players and it can be also simply extended to the case with inequality constraints. Since the players don't have access to other players' actions, they need to exchange estimates of others' actions and a local copy of the Lagrangian multiplier with their neighbors over a connected communication graph. We develop a relatively fast algorithm by reformulating the conservative GNE problem within the framework of inexact-ADMM. The convergence of the algorithm is guaranteed under a few mild assumptions on cost functions. Finally, the algorithm is simulated for a wireless ad-hoc network. △ Less

Submitted 24 March, 2017; originally announced March 2017.

Comments: arXiv admin note: text overlap with arXiv:1612.00414

A distributed primal-dual algorithm for computation of generalized Nash equilibria with shared affine coupling constraints via operator splitting methods

Authors: Peng Yi, Lacra Pavel

Abstract: In this paper, we propose a distributed primal-dual algorithm for computation of a generalized Nash equilibrium (GNE) in noncooperative games over network systems. In the considered game, not only each player's local objective function depends on other players' decisions, but also the feasible decision sets of all the players are coupled together with a globally shared affine inequality constraint… ▽ More In this paper, we propose a distributed primal-dual algorithm for computation of a generalized Nash equilibrium (GNE) in noncooperative games over network systems. In the considered game, not only each player's local objective function depends on other players' decisions, but also the feasible decision sets of all the players are coupled together with a globally shared affine inequality constraint. Adopting the variational GNE, that is the solution of a variational inequality, as a refinement of GNE, we introduce a primal-dual algorithm that players can use to seek it in a distributed manner. Each player only needs to know its local objective function, local feasible set, and a local block of the affine constraint. Meanwhile, each player only needs to observe the decisions on which its local objective function explicitly depends through the interference graph and share information related to multipliers with its neighbors through a multiplier graph. Through a primal-dual analysis and an augmentation of variables, we reformulate the problem as finding the zeros of a sum of monotone operators. Our distributed primal-dual algorithm is based on forward-backward operator splitting methods. We prove its convergence to the variational GNE for fixed step-sizes under some mild assumptions. Then a distributed algorithm with inertia is also introduced and analyzed for variational GNE seeking. Finally, numerical simulations for network Cournot competition are given to illustrate the algorithm efficiency and performance. △ Less

Submitted 15 March, 2017; originally announced March 2017.

Comments: 21 pages,8 figures, parts are submitted to IEEE CDC

Journal ref: Automatica, vol 102, pp. 111 -121, 2019

Nash Equilibrium Seeking with Non-doubly Stochastic Communication Weight Matrix

Authors: Farzad Salehisadaghiani, Lacra Pavel

Abstract: A distributed Nash equilibrium seeking algorithm is presented for networked games. We assume an incomplete information available to each player about the other players' actions. The players communicate over a strongly connected digraph to send/receive the estimates of the other players' actions to/from the other local players according to a gossip communication protocol. Due to asymmetric informat… ▽ More A distributed Nash equilibrium seeking algorithm is presented for networked games. We assume an incomplete information available to each player about the other players' actions. The players communicate over a strongly connected digraph to send/receive the estimates of the other players' actions to/from the other local players according to a gossip communication protocol. Due to asymmetric information exchange between the players, a non-doubly (row) stochastic weight matrix is defined. We show that, due to the non-doubly stochastic property, the total average of all players' estimates is not preserved for the next iteration which results in having no exact convergence. We present an almost sure convergence proof of the algorithm to a Nash equilibrium of the game. Then, we extend the algorithm for graphical games in which all players' cost functions are only dependent on the local neighboring players over an interference digraph. We design an assumption on the communication digraph such that the players are able to update all the estimates of the players who interfere with their cost functions. It is shown that the communication digraph needs to be a superset of a transitive reduction of the interference digraph. Finally, we verify the efficacy of the algorithm via a simulation on a social media behavioral case. △ Less

Submitted 30 March, 2017; v1 submitted 21 December, 2016; originally announced December 2016.

Journal ref: EAI Transactions on Collaborative Computing, 2019

Distributed Nash Equilibrium Seeking via the Alternating Direction Method of Multipliers

Authors: Farzad Salehisadaghiani, Lacra Pavel

Abstract: In this paper, the problem of finding a Nash equilibrium of a multi-player game is considered. The players are only aware of their own cost functions as well as the action space of all players. We develop a relatively fast algorithm within the framework of inexact-ADMM. It requires a communication graph for the information exchange between the players as well as a few mild assumptions on cost func… ▽ More In this paper, the problem of finding a Nash equilibrium of a multi-player game is considered. The players are only aware of their own cost functions as well as the action space of all players. We develop a relatively fast algorithm within the framework of inexact-ADMM. It requires a communication graph for the information exchange between the players as well as a few mild assumptions on cost functions. The convergence proof of the algorithm to a Nash equilibrium of the game is then provided. Moreover, the convergence rate is investigated via simulations. △ Less

Submitted 1 December, 2016; originally announced December 2016.

Distributed Nash Equilibrium Seeking By Gossip in Games on Graphs

Authors: Farzad Salehisadaghiani, Lacra Pavel

Abstract: We consider a gossip approach for finding a Nash equilibrium in a distributed multi-player network game. We extend previous results on Nash equilibrium seeking to the case when the players' cost functions may be affected by the actions of any subset of players. An interference graph is employed to illustrate the partially-coupled cost functions and the asymmetric information requirements. For a gi… ▽ More We consider a gossip approach for finding a Nash equilibrium in a distributed multi-player network game. We extend previous results on Nash equilibrium seeking to the case when the players' cost functions may be affected by the actions of any subset of players. An interference graph is employed to illustrate the partially-coupled cost functions and the asymmetric information requirements. For a given interference graph, we design a generalized communication graph so that players with possibly partially-coupled cost functions exchange only their required information and make decisions based on them. Using a set of standard assumptions on the cost functions, interference and communication graphs, we prove almost sure convergence to a Nash equilibrium for diminishing step sizes. We then quantify the effect of the second largest eigenvalue of the expected communication matrix on the convergence rate, and illustrate the trade-off between the parameters associated with the communication and the interference graphs. Finally, the efficacy of the proposed algorithm on a large-scale networked game is demonstrated via simulation. △ Less

Submitted 6 October, 2016; originally announced October 2016.

A Robust Mixed Integer Optimization Model to Utilize Regenerative Energy of Trains in a Railway Network

Authors: Shuvomoy Das Gupta, J. Kevin Tobin, Lacra Pavel

Abstract: In this paper we present a robust mixed integer optimization model to utilize regenerative braking energy produced by trains in a railway network. An electric train produces regenerative energy during braking, which is often lost in present technology. To utilize this energy we calculate a timetable which maximizes the total overlapping time between the braking and accelerating phases of suitable… ▽ More In this paper we present a robust mixed integer optimization model to utilize regenerative braking energy produced by trains in a railway network. An electric train produces regenerative energy during braking, which is often lost in present technology. To utilize this energy we calculate a timetable which maximizes the total overlapping time between the braking and accelerating phases of suitable train pairs, so that the regenerative energy of a braking train can be transferred to a suitable accelerating one. We apply our optimization model to different instances of a railway network for a time horizon spanning six hours. For each instance, our model finds an optimal timetable very quickly (largest running time being 86.64 seconds). Compared to the existing timetables, we observe significant increase in utilization of regenerative energy for every instance. △ Less

Submitted 6 July, 2015; originally announced July 2015.

Comments: 18 pages, 3 figures. arXiv admin note: substantial text overlap with arXiv:1506.08243

A Two-Step Linear Programming Model for Energy-Efficient Timetables in Metro Railway Networks

Authors: Shuvomoy Das Gupta, J. Kevin Tobin, Lacra Pavel

Abstract: In this paper we propose a novel two-step linear optimization model to calculate energy-efficient timetables in metro railway networks. The resultant timetable minimizes the total energy consumed by all trains and maximizes the utilization of regenerative energy produced by braking trains, subject to the constraints in the railway network. In contrast to other existing models, which are NP-hard, o… ▽ More In this paper we propose a novel two-step linear optimization model to calculate energy-efficient timetables in metro railway networks. The resultant timetable minimizes the total energy consumed by all trains and maximizes the utilization of regenerative energy produced by braking trains, subject to the constraints in the railway network. In contrast to other existing models, which are NP-hard, our model is computationally the most tractable one being a linear program. We apply our optimization model to different instances of service PES2-SFM2 of line 8 of Shanghai Metro network spanning a full service period of one day (18 hours) with thousands of active trains. For every instance, our model finds an optimal timetable very quickly (largest runtime being less than 13s) with significant reduction in effective energy consumption (the worst case being 19.27%). Code based on the model has been integrated with Thales Timetable Compiler - the industrial timetable compiler of Thales Inc that has the largest installed base of communication-based train control systems worldwide. △ Less

Submitted 29 May, 2018; v1 submitted 26 June, 2015; originally announced June 2015.

Comments: 25 pages, 7 figures

Journal ref: Transportation Research Part B: Methodological 93 (2016): 57-74

Enabling Differentiated Services Using Generalized Power Control Model in Optical Networks

Authors: Quanyan Zhu, Lacra Pavel

Abstract: This paper considers a generalized framework to study OSNR optimization-based end-to-end link level power control problems in optical networks. We combine favorable features of game-theoretical approach and central cost approach to allow different service groups within the network. We develop solutions concepts for both cases of empty and nonempty feasible sets. In addition, we derive and prove th… ▽ More This paper considers a generalized framework to study OSNR optimization-based end-to-end link level power control problems in optical networks. We combine favorable features of game-theoretical approach and central cost approach to allow different service groups within the network. We develop solutions concepts for both cases of empty and nonempty feasible sets. In addition, we derive and prove the convergence of a distributed iterative algorithm for different classes of users. In the end, we use numerical examples to illustrate the novel framework. △ Less

Submitted 12 March, 2011; originally announced March 2011.

An Optimization and Control Theoretic Approach to Noncooperative Game Design

Authors: Tansu Alpcan, Lacra Pavel, Nem Stefanovic

Abstract: This paper investigates design of noncooperative games from an optimization and control theoretic perspective. Pricing mechanisms are used as a design tool to ensure that the Nash equilibrium of a fairly general class of noncooperative games satisfies certain global objectives such as welfare maximization or achieving a certain level of quality-of-service (QoS). The class of games considered provi… ▽ More This paper investigates design of noncooperative games from an optimization and control theoretic perspective. Pricing mechanisms are used as a design tool to ensure that the Nash equilibrium of a fairly general class of noncooperative games satisfies certain global objectives such as welfare maximization or achieving a certain level of quality-of-service (QoS). The class of games considered provide a theoretical basis for decentralized resource allocation and control problems including network congestion control, wireless uplink power control, and optical power control. The game design problem is analyzed under different knowledge assumptions (full versus limited information) and design objectives (QoS versus utility maximization) for separable and non-separable utility functions. The ``price of anarchy'' is shown not to be an inherent feature of full-information games that incorporate pricing mechanisms. Moreover, a simple linear pricing is shown to be sufficient for design of Nash equilibrium according to a chosen global objective for a fairly general class of games. Stability properties of the game and pricing dynamics are studied under the assumption of time-scale separation and in two separate time-scales. Thus, sufficient conditions are derived, which allow the designer to place the Nash equilibrium solution or to guide the system trajectory to a desired region or point. The obtained results are illustrated with a number of examples. △ Less

Submitted 1 July, 2010; originally announced July 2010.

Comments: Earlier versions of this work have appeared partly in the International Conference on Game Theory for Networks (GameNets), in Istanbul, Turkey in May 2009 and Conference on Decision and Control (CDC), Shanghai, China, in December 2009