Skip to main content

Showing 1–50 of 125 results for author: Piliouras, G

.
  1. arXiv:2506.05005  [pdf, ps, other

    cs.LG cs.GT math.OC

    Cautious Optimism: A Meta-Algorithm for Near-Constant Regret in General Games

    Authors: Ashkan Soleymani, Georgios Piliouras, Gabriele Farina

    Abstract: Recent work [Soleymani et al., 2025] introduced a variant of Optimistic Multiplicative Weights Updates (OMWU) that adaptively controls the learning pace in a dynamic, non-monotone manner, achieving new state-of-the-art regret minimization guarantees in general games. In this work, we demonstrate that no-regret learning acceleration through adaptive pacing of the learners is not an isolated phenome… ▽ More

    Submitted 5 June, 2025; originally announced June 2025.

    Comments: Extended abstract appeared at Twenty-Sixth ACM Conference on Economics and Computation (EC), 2025

  2. arXiv:2505.10361  [pdf, other

    cs.AI cs.LG

    Plasticity as the Mirror of Empowerment

    Authors: David Abel, Michael Bowling, André Barreto, Will Dabney, Shi Dong, Steven Hansen, Anna Harutyunyan, Khimya Khetarpal, Clare Lyle, Razvan Pascanu, Georgios Piliouras, Doina Precup, Jonathan Richens, Mark Rowland, Tom Schaul, Satinder Singh

    Abstract: Agents are minimally entities that are influenced by their past observations and act to influence future observations. This latter capacity is captured by empowerment, which has served as a vital framing concept across artificial intelligence and cognitive science. This former capacity, however, is equally foundational: In what ways, and to what extent, can an agent be influenced by what it observ… ▽ More

    Submitted 15 May, 2025; originally announced May 2025.

  3. arXiv:2503.24340  [pdf, ps, other

    cs.GT cs.LG math.OC

    Faster Rates for No-Regret Learning in General Games via Cautious Optimism

    Authors: Ashkan Soleymani, Georgios Piliouras, Gabriele Farina

    Abstract: We establish the first uncoupled learning algorithm that attains $O(n \log^2 d \log T)$ per-player regret in multi-player general-sum games, where $n$ is the number of players, $d$ is the number of actions available to each player, and $T$ is the number of repetitions of the game. Our results exponentially improve the dependence on $d$ compared to the $O(n\, d \log T)$ regret attainable by Log-Reg… ▽ More

    Submitted 31 March, 2025; originally announced March 2025.

    Comments: Appeared at STOC 2025

  4. arXiv:2503.22850  [pdf, other

    eess.SY

    Passivity, No-Regret, and Convergent Learning in Contractive Games

    Authors: Hassan Abdelraouf, Georgios Piliouras, Jeff S. Shamma

    Abstract: We investigate the interplay between passivity, no-regret, and convergence in contractive games for various learning dynamic models and their higher-order variants. Our setting is continuous time. Building on prior work for replicator dynamics, we show that if learning dynamics satisfy a passivity condition between the payoff vector and the difference between its evolving strategy and any fixed st… ▽ More

    Submitted 28 March, 2025; originally announced March 2025.

  5. arXiv:2502.20170  [pdf, other

    cs.GT cs.CL cs.LG stat.ML

    Re-evaluating Open-ended Evaluation of Large Language Models

    Authors: Siqi Liu, Ian Gemp, Luke Marris, Georgios Piliouras, Nicolas Heess, Marc Lanctot

    Abstract: Evaluation has traditionally focused on ranking candidates for a specific skill. Modern generalist models, such as Large Language Models (LLMs), decidedly outpace this paradigm. Open-ended evaluation systems, where candidate models are compared on user-submitted prompts, have emerged as a popular solution. Despite their many advantages, we show that the current Elo-based rating systems can be susc… ▽ More

    Submitted 8 May, 2025; v1 submitted 27 February, 2025; originally announced February 2025.

    Comments: Published at ICLR 2025

  6. arXiv:2502.14143  [pdf, other

    cs.MA cs.AI cs.CY cs.ET cs.LG

    Multi-Agent Risks from Advanced AI

    Authors: Lewis Hammond, Alan Chan, Jesse Clifton, Jason Hoelscher-Obermaier, Akbir Khan, Euan McLean, Chandler Smith, Wolfram Barfuss, Jakob Foerster, Tomáš Gavenčiak, The Anh Han, Edward Hughes, Vojtěch Kovařík, Jan Kulveit, Joel Z. Leibo, Caspar Oesterheld, Christian Schroeder de Witt, Nisarg Shah, Michael Wellman, Paolo Bova, Theodor Cimpeanu, Carson Ezell, Quentin Feuillade-Montixi, Matija Franklin, Esben Kran , et al. (19 additional authors not shown)

    Abstract: The rapid development of advanced AI agents and the imminent deployment of many instances of these agents will give rise to multi-agent systems of unprecedented complexity. These systems pose novel and under-explored risks. In this report, we provide a structured taxonomy of these risks by identifying three key failure modes (miscoordination, conflict, and collusion) based on agents' incentives, a… ▽ More

    Submitted 19 February, 2025; originally announced February 2025.

    Comments: Cooperative AI Foundation, Technical Report #1

  7. arXiv:2502.11645  [pdf, other

    cs.GT cs.CL cs.MA stat.OT

    Deviation Ratings: A General, Clone-Invariant Rating Method

    Authors: Luke Marris, Siqi Liu, Ian Gemp, Georgios Piliouras, Marc Lanctot

    Abstract: Many real-world multi-agent or multi-task evaluation scenarios can be naturally modelled as normal-form games due to inherent strategic (adversarial, cooperative, and mixed motive) interactions. These strategic interactions may be agentic (e.g. players trying to win), fundamental (e.g. cost vs quality), or complementary (e.g. niche finding and specialization). In such a formulation, it is the stra… ▽ More

    Submitted 17 February, 2025; originally announced February 2025.

  8. arXiv:2502.04403  [pdf, other

    cs.AI

    Agency Is Frame-Dependent

    Authors: David Abel, André Barreto, Michael Bowling, Will Dabney, Shi Dong, Steven Hansen, Anna Harutyunyan, Khimya Khetarpal, Clare Lyle, Razvan Pascanu, Georgios Piliouras, Doina Precup, Jonathan Richens, Mark Rowland, Tom Schaul, Satinder Singh

    Abstract: Agency is a system's capacity to steer outcomes toward a goal, and is a central topic of study across biology, philosophy, cognitive science, and artificial intelligence. Determining if a system exhibits agency is a notoriously difficult question: Dennett (1989), for instance, highlights the puzzle of determining which principles can decide whether a rock, a thermostat, or a robot each possess age… ▽ More

    Submitted 6 February, 2025; originally announced February 2025.

  9. arXiv:2412.20203  [pdf, other

    cs.GT cs.LG cs.MA math.OC

    No-regret learning in harmonic games: Extrapolation in the face of conflicting interests

    Authors: Davide Legacci, Panayotis Mertikopoulos, Christos H. Papadimitriou, Georgios Piliouras, Bary S. R. Pradelski

    Abstract: The long-run behavior of multi-agent learning - and, in particular, no-regret learning - is relatively well-understood in potential games, where players have aligned interests. By contrast, in harmonic games - the strategic counterpart of potential games, where players have conflicting interests - very little is known outside the narrow subclass of 2-player zero-sum games with a fully-mixed equili… ▽ More

    Submitted 28 December, 2024; originally announced December 2024.

    Comments: 36 pages, 5 figures

    MSC Class: Primary 91A10; 91A26; secondary 68Q32; 68T02

  10. arXiv:2412.19010  [pdf, other

    cs.AI

    A theory of appropriateness with applications to generative artificial intelligence

    Authors: Joel Z. Leibo, Alexander Sasha Vezhnevets, Manfred Diaz, John P. Agapiou, William A. Cunningham, Peter Sunehag, Julia Haas, Raphael Koster, Edgar A. Duéñez-Guzmán, William S. Isaac, Georgios Piliouras, Stanley M. Bileschi, Iyad Rahwan, Simon Osindero

    Abstract: What is appropriateness? Humans navigate a multi-scale mosaic of interlocking notions of what is appropriate for different situations. We act one way with our friends, another with our family, and yet another in the office. Likewise for AI, appropriate behavior for a comedy-writing assistant is not the same as appropriate behavior for a customer-service representative. What determines which action… ▽ More

    Submitted 25 December, 2024; originally announced December 2024.

    Comments: 115 pages, 2 figures

  11. arXiv:2412.05747  [pdf, other

    cs.GT cs.AI

    Charting the Shapes of Stories with Game Theory

    Authors: Constantinos Daskalakis, Ian Gemp, Yanchen Jiang, Renato Paes Leme, Christos Papadimitriou, Georgios Piliouras

    Abstract: Stories are records of our experiences and their analysis reveals insights into the nature of being human. Successful analyses are often interdisciplinary, leveraging mathematical tools to extract structure from stories and insights from structure. Historically, these tools have been restricted to one dimensional charts and dynamic social networks; however, modern AI offers the possibility of iden… ▽ More

    Submitted 7 December, 2024; originally announced December 2024.

    Comments: NeurIPS 2024 Creative AI Track

  12. arXiv:2411.01495  [pdf, other

    math.DS

    Interval maps mimicking circle rotations

    Authors: Jakub Bielawski, Thiparat Chotibut, Fryderyk Falniowski, Michał Misiurewicz, Georgios Piliouras

    Abstract: We investigate the dynamics of maps of the real line whose behavior on an invariant interval is close to a rational rotation on the circle. We concentrate on a specific two-parameter family, describing the dynamics arising from models in game theory, mathematical biology and machine learning. If one parameter is a rational number, $k/n$, with $k,n$ coprime, and the second one is large enough, we p… ▽ More

    Submitted 3 November, 2024; originally announced November 2024.

    MSC Class: 37E05

  13. arXiv:2410.16600  [pdf, other

    cs.GT cs.AI cs.MA

    Convex Markov Games: A Framework for Creativity, Imitation, Fairness, and Safety in Multiagent Learning

    Authors: Ian Gemp, Andreas Haupt, Luke Marris, Siqi Liu, Georgios Piliouras

    Abstract: Behavioral diversity, expert imitation, fairness, safety goals and others give rise to preferences in sequential decision making domains that do not decompose additively across time. We introduce the class of convex Markov games that allow general convex preferences over occupancy measures. Despite infinite time horizon and strictly higher generality than Markov games, pure strategy Nash equilibri… ▽ More

    Submitted 16 January, 2025; v1 submitted 21 October, 2024; originally announced October 2024.

  14. arXiv:2408.11146  [pdf, other

    cs.GT cs.LG econ.TH

    Swim till You Sink: Computing the Limit of a Game

    Authors: Rashida Hakim, Jason Milionis, Christos Papadimitriou, Georgios Piliouras

    Abstract: During 2023, two interesting results were proven about the limit behavior of game dynamics: First, it was shown that there is a game for which no dynamics converges to the Nash equilibria. Second, it was shown that the sink equilibria of a game adequately capture the limit behavior of natural game dynamics. These two results have created a need and opportunity to articulate a principled computatio… ▽ More

    Submitted 20 August, 2024; originally announced August 2024.

  15. arXiv:2408.07685  [pdf, ps, other

    cs.GT

    Auto-bidding and Auctions in Online Advertising: A Survey

    Authors: Gagan Aggarwal, Ashwinkumar Badanidiyuru, Santiago R. Balseiro, Kshipra Bhawalkar, Yuan Deng, Zhe Feng, Gagan Goel, Christopher Liaw, Haihao Lu, Mohammad Mahdian, Jieming Mao, Aranyak Mehta, Vahab Mirrokni, Renato Paes Leme, Andres Perlroth, Georgios Piliouras, Jon Schneider, Ariel Schvartzman, Balasubramanian Sivan, Kelly Spendlove, Yifeng Teng, Di Wang, Hanrui Zhang, Mingfei Zhao, Wennan Zhu , et al. (1 additional authors not shown)

    Abstract: In this survey, we summarize recent developments in research fueled by the growing adoption of automated bidding strategies in online advertising. We explore the challenges and opportunities that have arisen as markets embrace this autobidding and cover a range of topics in this area, including bidding algorithms, equilibrium analysis and efficiency of common auction formats, and optimal auction d… ▽ More

    Submitted 14 August, 2024; originally announced August 2024.

  16. arXiv:2406.19350  [pdf, other

    cs.GT

    Complex Dynamics in Autobidding Systems

    Authors: Renato Paes Leme, Georgios Piliouras, Jon Schneider, Kelly Spendlove, Song Zuo

    Abstract: It has become the default in markets such as ad auctions for participants to bid in an auction through automated bidding agents (autobidders) which adjust bids over time to satisfy return-over-spend constraints. Despite the prominence of such systems for the internet economy, their resulting dynamical behavior is still not well understood. Although one might hope that such relatively simple system… ▽ More

    Submitted 1 July, 2024; v1 submitted 27 June, 2024; originally announced June 2024.

  17. arXiv:2406.10603  [pdf, other

    cs.GT

    Prediction Accuracy of Learning in Games : Follow-the-Regularized-Leader meets Heisenberg

    Authors: Yi Feng, Georgios Piliouras, Xiao Wang

    Abstract: We investigate the accuracy of prediction in deterministic learning dynamics of zero-sum games with random initializations, specifically focusing on observer uncertainty and its relationship to the evolution of covariances. Zero-sum games are a prominent field of interest in machine learning due to their various applications. Concurrently, the accuracy of prediction in dynamical systems from mecha… ▽ More

    Submitted 15 June, 2024; originally announced June 2024.

    Comments: Accepted for ICML 2024

  18. arXiv:2404.01066  [pdf, ps, other

    eess.SY cs.GT

    Learning and steering game dynamics towards desirable outcomes

    Authors: Ilayda Canyakmaz, Iosif Sakos, Wayne Lin, Antonios Varvitsiotis, Georgios Piliouras

    Abstract: Game dynamics, which describe how agents' strategies evolve over time based on past interactions, can exhibit a variety of undesirable behaviours including convergence to suboptimal equilibria, cycling, and chaos. While central planners can employ incentives to mitigate such behaviors and steer game dynamics towards desirable outcomes, the effectiveness of such interventions critically relies on a… ▽ More

    Submitted 10 December, 2024; v1 submitted 1 April, 2024; originally announced April 2024.

  19. arXiv:2403.15848  [pdf, other

    cs.GT

    On the Stability of Learning in Network Games with Many Players

    Authors: Aamal Hussain, Dan Leonte, Francesco Belardinelli, Georgios Piliouras

    Abstract: Multi-agent learning algorithms have been shown to display complex, unstable behaviours in a wide array of games. In fact, previous works indicate that convergent behaviours are less likely to occur as the total number of agents increases. This seemingly prohibits convergence to stable strategies, such as Nash Equilibria, in games with many players. To make progress towards addressing this chall… ▽ More

    Submitted 23 March, 2024; originally announced March 2024.

    Comments: AAMAS 2024. arXiv admin note: text overlap with arXiv:2307.13922

    MSC Class: 93A16; 91A26; 91A68; 58K35 ACM Class: G.3; J.4; F.2.2

  20. arXiv:2402.16985  [pdf, other

    cs.GT cs.SE

    Visualizing 2x2 Normal-Form Games: twoxtwogame LaTeX Package

    Authors: Luke Marris, Ian Gemp, Siqi Liu, Joel Z. Leibo, Georgios Piliouras

    Abstract: Normal-form games with two players, each with two strategies, are the most studied class of games. These so-called 2x2 games are used to model a variety of strategic interactions. They appear in game theory, economics, and artificial intelligence research. However, there lacks tools for describing and visualizing such games. This work introduces a LaTeX package for visualizing 2x2 games. This work… ▽ More

    Submitted 26 February, 2024; originally announced February 2024.

  21. arXiv:2402.15849  [pdf, other

    cs.GT econ.TH math.DS

    MEV Sharing with Dynamic Extraction Rates

    Authors: Pedro Braga, Georgios Chionas, Piotr Krysta, Stefanos Leonardos, Georgios Piliouras, Carmine Ventre

    Abstract: Maximal Extractable Value (MEV) has emerged as a new frontier in the design of blockchain systems. In this paper, we propose making the MEV extraction rate as part of the protocol design space. Our aim is to leverage this parameter to maintain a healthy balance between block producers (who need to be compensated) and users (who need to feel encouraged to transact). We follow the approach introduce… ▽ More

    Submitted 30 September, 2024; v1 submitted 24 February, 2024; originally announced February 2024.

    Comments: Extended abstract in the 23rd International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS 2024)

    MSC Class: 37; 65P20; 91; 93A14; 93A16 ACM Class: C.2.4; F.2; I.2.11; J.2; J.4

  22. arXiv:2402.08393  [pdf, other

    cs.GT

    NfgTransformer: Equivariant Representation Learning for Normal-form Games

    Authors: Siqi Liu, Luke Marris, Georgios Piliouras, Ian Gemp, Nicolas Heess

    Abstract: Normal-form games (NFGs) are the fundamental model of strategic interaction. We study their representation using neural networks. We describe the inherent equivariance of NFGs -- any permutation of strategies describes an equivalent game -- as well as the challenges this poses for representation learning. We then propose the NfgTransformer architecture that leverages this equivariance, leading to… ▽ More

    Submitted 13 February, 2024; originally announced February 2024.

    Comments: Published at ICLR 2024. Open-sourced at https://github.com/google-deepmind/nfg_transformer

  23. arXiv:2402.03928  [pdf, other

    cs.GT cs.MA

    Approximating the Core via Iterative Coalition Sampling

    Authors: Ian Gemp, Marc Lanctot, Luke Marris, Yiran Mao, Edgar Duéñez-Guzmán, Sarah Perrin, Andras Gyorgy, Romuald Elie, Georgios Piliouras, Michael Kaisers, Daniel Hennes, Kalesha Bullard, Kate Larson, Yoram Bachrach

    Abstract: The core is a central solution concept in cooperative game theory, defined as the set of feasible allocations or payments such that no subset of agents has incentive to break away and form their own subgroup or coalition. However, it has long been known that the core (and approximations, such as the least-core) are hard to compute. This limits our ability to analyze cooperative games in general, a… ▽ More

    Submitted 6 February, 2024; originally announced February 2024.

    Comments: Published in AAMAS 2024

  24. arXiv:2402.01704  [pdf, other

    cs.CL cs.AI cs.GT

    Steering Language Models with Game-Theoretic Solvers

    Authors: Ian Gemp, Roma Patel, Yoram Bachrach, Marc Lanctot, Vibhavari Dasagi, Luke Marris, Georgios Piliouras, Siqi Liu, Karl Tuyls

    Abstract: Mathematical models of interactions among rational agents have long been studied in game theory. However these interactions are often over a small set of discrete game actions which is very different from how humans communicate in natural language. To bridge this gap, we introduce a framework that allows equilibrium solvers to work over the space of natural language dialogue generated by large lan… ▽ More

    Submitted 16 December, 2024; v1 submitted 24 January, 2024; originally announced February 2024.

    Comments: Code available @ https://github.com/google-deepmind/open_spiel/blob/master/open_spiel/python/games/chat_game.py

  25. arXiv:2401.05133  [pdf, other

    cs.AI cs.MA

    Neural Population Learning beyond Symmetric Zero-sum Games

    Authors: Siqi Liu, Luke Marris, Marc Lanctot, Georgios Piliouras, Joel Z. Leibo, Nicolas Heess

    Abstract: We study computationally efficient methods for finding equilibria in n-player general-sum games, specifically ones that afford complex visuomotor skills. We show how existing methods would struggle in this setting, either computationally or in theory. We then introduce NeuPL-JPSRO, a neural population learning algorithm that benefits from transfer learning of skills and converges to a Coarse Corre… ▽ More

    Submitted 10 January, 2024; originally announced January 2024.

  26. arXiv:2312.16609  [pdf, other

    cs.GT cs.LG

    Exploiting hidden structures in non-convex games for convergence to Nash equilibrium

    Authors: Iosif Sakos, Emmanouil-Vasileios Vlatakis-Gkaragkounis, Panayotis Mertikopoulos, Georgios Piliouras

    Abstract: A wide array of modern machine learning applications - from adversarial models to multi-agent reinforcement learning - can be formulated as non-cooperative games whose Nash equilibria represent the system's desired operational states. Despite having a highly non-convex loss landscape, many cases of interest possess a latent convex structure that could potentially be leveraged to yield convergence… ▽ More

    Submitted 27 December, 2023; originally announced December 2023.

    Comments: 32 pages, 18 figures

    MSC Class: Primary 91A10; 91A26; secondary 68Q32

  27. arXiv:2311.14125  [pdf, other

    cs.AI cs.LG

    Scalable AI Safety via Doubly-Efficient Debate

    Authors: Jonah Brown-Cohen, Geoffrey Irving, Georgios Piliouras

    Abstract: The emergence of pre-trained AI systems with powerful capabilities across a diverse and ever-increasing set of complex domains has raised a critical challenge for AI safety as tasks can become too complicated for humans to judge directly. Irving et al. [2018] proposed a debate method in this direction with the goal of pitting the power of such AI models against each other until the problem of iden… ▽ More

    Submitted 23 November, 2023; originally announced November 2023.

  28. arXiv:2311.10859  [pdf, other

    quant-ph cs.GT cs.LG math.OC

    A Quadratic Speedup in Finding Nash Equilibria of Quantum Zero-Sum Games

    Authors: Francisca Vasconcelos, Emmanouil-Vasileios Vlatakis-Gkaragkounis, Panayotis Mertikopoulos, Georgios Piliouras, Michael I. Jordan

    Abstract: Recent developments in domains such as non-local games, quantum interactive proofs, and quantum generative adversarial networks have renewed interest in quantum game theory and, specifically, quantum zero-sum games. Central to classical game theory is the efficient algorithmic computation of Nash equilibria, which represent optimal strategies for both players. In 2008, Jain and Watrous proposed th… ▽ More

    Submitted 2 May, 2025; v1 submitted 17 November, 2023; originally announced November 2023.

    Comments: 53 pages, 7 figures, QTML 2023 (Long Talk), Quantum Journal 2025

    MSC Class: primary 91A05; 81Q93; secondary 68Q32; 91A26; 37N40;

    Journal ref: Quantum 9, 1737 (2025)

  29. No-Regret Learning and Equilibrium Computation in Quantum Games

    Authors: Wayne Lin, Georgios Piliouras, Ryann Sim, Antonios Varvitsiotis

    Abstract: As quantum processors advance, the emergence of large-scale decentralized systems involving interacting quantum-enabled agents is on the horizon. Recent research efforts have explored quantum versions of Nash and correlated equilibria as solution concepts of strategic quantum interactions, but these approaches did not directly connect to decentralized adaptive setups where agents possess limited i… ▽ More

    Submitted 1 December, 2024; v1 submitted 12 October, 2023; originally announced October 2023.

    Journal ref: Quantum 8, 1569 (2024)

  30. arXiv:2310.06689  [pdf, other

    cs.GT cs.MA

    Approximating Nash Equilibria in Normal-Form Games via Stochastic Optimization

    Authors: Ian Gemp, Luke Marris, Georgios Piliouras

    Abstract: We propose the first loss function for approximate Nash equilibria of normal-form games that is amenable to unbiased Monte Carlo estimation. This construction allows us to deploy standard non-convex stochastic optimization techniques for approximating Nash equilibria, resulting in novel algorithms with provable guarantees. We complement our theoretical analysis with experiments demonstrating that… ▽ More

    Submitted 15 April, 2024; v1 submitted 10 October, 2023; originally announced October 2023.

    Comments: Published at ICLR 2024

  31. arXiv:2307.13928  [pdf, other

    cs.GT

    Beyond Strict Competition: Approximate Convergence of Multi Agent Q-Learning Dynamics

    Authors: Aamal Hussain, Francesco Belardinelli, Georgios Piliouras

    Abstract: The behaviour of multi-agent learning in competitive settings is often considered under the restrictive assumption of a zero-sum game. Only under this strict requirement is the behaviour of learning well understood; beyond this, learning dynamics can often display non-convergent behaviours which prevent fixed-point analysis. Nonetheless, many relevant competitive games do not satisfy the zero-sum… ▽ More

    Submitted 25 July, 2023; originally announced July 2023.

    Comments: Presented at IJCAI 2023

    MSC Class: 93A16; 91A26; 91A68; 58K35 ACM Class: G.3; J.4; F.2.2

  32. arXiv:2307.13922  [pdf, other

    cs.GT cs.AI cs.MA math.DS

    Stability of Multi-Agent Learning: Convergence in Network Games with Many Players

    Authors: Aamal Hussain, Dan Leonte, Francesco Belardinelli, Georgios Piliouras

    Abstract: The behaviour of multi-agent learning in many player games has been shown to display complex dynamics outside of restrictive examples such as network zero-sum games. In addition, it has been shown that convergent behaviour is less likely to occur as the number of players increase. To make progress in resolving this problem, we study Q-Learning dynamics and determine a sufficient condition for the… ▽ More

    Submitted 25 July, 2023; originally announced July 2023.

    Comments: Presented at the Workshop on New Frontiers in Learning, Control, and Dynamical Systems at the International Conference on Machine Learning (ICML), Honolulu, Hawaii, USA, 2023

    MSC Class: 93A16; 91A26; 91A68; 58K35 ACM Class: G.3; J.4; F.2.2

  33. arXiv:2307.06640  [pdf, other

    cs.GT cs.LG math.OC

    Data-Scarce Identification of Game Dynamics via Sum-of-Squares Optimization

    Authors: Iosif Sakos, Antonios Varvitsiotis, Georgios Piliouras

    Abstract: Understanding how players adjust their strategies in games, based on their experience, is a crucial tool for policymakers. It enables them to forecast the system's eventual behavior, exert control over the system, and evaluate counterfactual scenarios. The task becomes increasingly difficult when only a limited number of observations are available or difficult to acquire. In this work, we introduc… ▽ More

    Submitted 11 October, 2024; v1 submitted 13 July, 2023; originally announced July 2023.

  34. arXiv:2307.03136  [pdf, other

    math.OC cs.LG stat.ML

    Multiplicative Updates for Online Convex Optimization over Symmetric Cones

    Authors: Ilayda Canyakmaz, Wayne Lin, Georgios Piliouras, Antonios Varvitsiotis

    Abstract: We study online convex optimization where the possible actions are trace-one elements in a symmetric cone, generalizing the extensively-studied experts setup and its quantum counterpart. Symmetric cones provide a unifying framework for some of the most important optimization models, including linear, second-order cone, and semidefinite optimization. Using tools from the field of Euclidean Jordan A… ▽ More

    Submitted 6 July, 2023; originally announced July 2023.

    Comments: 27 pages, 7 figures, 2 tables

  35. arXiv:2306.01032  [pdf, other

    cs.LG math.OC

    Chaos persists in large-scale multi-agent learning despite adaptive learning rates

    Authors: Emmanouil-Vasileios Vlatakis-Gkaragkounis, Lampros Flokas, Georgios Piliouras

    Abstract: Multi-agent learning is intrinsically harder, more unstable and unpredictable than single agent optimization. For this reason, numerous specialized heuristics and techniques have been designed towards the goal of achieving convergence to equilibria in self-play. One such celebrated approach is the use of dynamically adaptive learning rates. Although such techniques are known to allow for improved… ▽ More

    Submitted 1 June, 2023; originally announced June 2023.

    Comments: 30 pages, 6 figures

  36. arXiv:2304.09978  [pdf, other

    cs.GT cs.MA econ.TH math.OC

    Equilibrium-Invariant Embedding, Metric Space, and Fundamental Set of $2\times2$ Normal-Form Games

    Authors: Luke Marris, Ian Gemp, Georgios Piliouras

    Abstract: Equilibrium solution concepts of normal-form games, such as Nash equilibria, correlated equilibria, and coarse correlated equilibria, describe the joint strategy profiles from which no player has incentive to unilaterally deviate. They are widely studied in game theory, economics, and multiagent systems. Equilibrium concepts are invariant under certain transforms of the payoffs. We define an equil… ▽ More

    Submitted 19 April, 2023; originally announced April 2023.

    Comments: 42 pages

  37. A stochastic variant of replicator dynamics in zero-sum games and its invariant measures

    Authors: Maximilian Engel, Georgios Piliouras

    Abstract: We study the behavior of a stochastic variant of replicator dynamics in two-agent zero-sum games. We characterize the statistics of such systems by their invariant measures which can be shown to be entirely supported on the boundary of the space of mixed strategies. Depending on the noise strength we can furthermore characterize these invariant measures by finding accumulation of mass at specific… ▽ More

    Submitted 25 October, 2023; v1 submitted 14 February, 2023; originally announced February 2023.

    MSC Class: 60H10; 60J70; 91A15; 91A22; 91A25; 91A68

  38. arXiv:2302.06607  [pdf, other

    cs.GT

    Generative Adversarial Equilibrium Solvers

    Authors: Denizalp Goktas, David C. Parkes, Ian Gemp, Luke Marris, Georgios Piliouras, Romuald Elie, Guy Lever, Andrea Tacchetti

    Abstract: We introduce the use of generative adversarial learning to compute equilibria in general game-theoretic settings, specifically the generalized Nash equilibrium (GNE) in pseudo-games, and its specific instantiation as the competitive equilibrium (CE) in Arrow-Debreu competitive economies. Pseudo-games are a generalization of games in which players' actions affect not only the payoffs of other playe… ▽ More

    Submitted 20 February, 2023; v1 submitted 13 February, 2023; originally announced February 2023.

    Comments: 41 pages, 13 figures

  39. Learning in Quantum Common-Interest Games and the Separability Problem

    Authors: Wayne Lin, Georgios Piliouras, Ryann Sim, Antonios Varvitsiotis

    Abstract: Learning in games has emerged as a powerful tool for machine learning with numerous applications. Quantum games model interactions between strategic players who have access to quantum resources, and several recent works have studied {learning in} the competitive regime of quantum zero-sum games. Going beyond this setting, we introduce quantum common-interest games (CIGs) where players have density… ▽ More

    Submitted 30 March, 2025; v1 submitted 9 February, 2023; originally announced February 2023.

    Journal ref: Quantum 9, 1689 (2025)

  40. arXiv:2301.09619  [pdf, other

    cs.GT cs.AI cs.MA math.DS

    Asymptotic Convergence and Performance of Multi-Agent Q-Learning Dynamics

    Authors: Aamal Abbas Hussain, Francesco Belardinelli, Georgios Piliouras

    Abstract: Achieving convergence of multiple learning agents in general $N$-player games is imperative for the development of safe and reliable machine learning (ML) algorithms and their application to autonomous systems. Yet it is known that, outside the bounds of simple two-player games, convergence cannot be taken for granted. To make progress in resolving this problem, we study the dynamics of smooth Q… ▽ More

    Submitted 23 January, 2023; originally announced January 2023.

    Comments: Accepted in AAMAS 2023

    MSC Class: 93A16; 91A26; 91A68; 58K35 ACM Class: G.3; J.4; F.2.2

  41. arXiv:2301.04929  [pdf, other

    cs.MA

    Heterogeneous Beliefs and Multi-Population Learning in Network Games

    Authors: Shuyue Hu, Harold Soh, Georgios Piliouras

    Abstract: The effect of population heterogeneity in multi-agent learning is practically relevant but remains far from being well-understood. Motivated by this, we introduce a model of multi-population learning that allows for heterogeneous beliefs within each population and where agents respond to their beliefs via smooth fictitious play (SFP).We show that the system state -- a probability distribution over… ▽ More

    Submitted 12 January, 2023; originally announced January 2023.

  42. arXiv:2301.03931  [pdf, ps, other

    cs.GT cs.LG math.OC

    Min-Max Optimization Made Simple: Approximating the Proximal Point Method via Contraction Maps

    Authors: Volkan Cevher, Georgios Piliouras, Ryann Sim, Stratis Skoulakis

    Abstract: In this paper we present a first-order method that admits near-optimal convergence rates for convex/concave min-max problems while requiring a simple and intuitive analysis. Similarly to the seminal work of Nemirovski and the recent approach of Piliouras et al. in normal form games, our work is based on the fact that the update rule of the Proximal Point method (PP) can be approximated up to accur… ▽ More

    Submitted 16 January, 2023; v1 submitted 10 January, 2023; originally announced January 2023.

    Comments: To appear in SOSA23

  43. arXiv:2212.07175  [pdf, other

    cs.GT cs.CR

    Optimality Despite Chaos in Fee Markets

    Authors: Stefanos Leonardos, Daniël Reijsbergen, Barnabé Monnot, Georgios Piliouras

    Abstract: Transaction fee markets are essential components of blockchain economies, as they resolve the inherent scarcity in the number of transactions that can be added to each block. In early blockchain protocols, this scarcity was resolved through a first-price auction in which users were forced to guess appropriate bids from recent blockchain data. Ethereum's EIP-1559 fee market reform streamlines this… ▽ More

    Submitted 15 December, 2022; v1 submitted 14 December, 2022; originally announced December 2022.

    MSC Class: 91A80; 91-10; 91B26

  44. arXiv:2211.01681  [pdf, other

    cs.GT quant-ph

    Matrix Multiplicative Weights Updates in Quantum Zero-Sum Games: Conservation Laws & Recurrence

    Authors: Rahul Jain, Georgios Piliouras, Ryann Sim

    Abstract: Recent advances in quantum computing and in particular, the introduction of quantum GANs, have led to increased interest in quantum zero-sum game theory, extending the scope of learning algorithms for classical games into the quantum realm. In this paper, we focus on learning in quantum zero-sum games under Matrix Multiplicative Weights Update (a generalization of the multiplicative weights update… ▽ More

    Submitted 26 April, 2023; v1 submitted 3 November, 2022; originally announced November 2022.

    Comments: NeurIPS 2022

  45. arXiv:2208.10138  [pdf, other

    cs.GT stat.ML

    Learning Correlated Equilibria in Mean-Field Games

    Authors: Paul Muller, Romuald Elie, Mark Rowland, Mathieu Lauriere, Julien Perolat, Sarah Perrin, Matthieu Geist, Georgios Piliouras, Olivier Pietquin, Karl Tuyls

    Abstract: The designs of many large-scale systems today, from traffic routing environments to smart grids, rely on game-theoretic equilibrium concepts. However, as the size of an $N$-player game typically grows exponentially with $N$, standard game theoretic analysis becomes effectively infeasible beyond a low number of players. Recent approaches have gone around this limitation by instead considering Mean-… ▽ More

    Submitted 22 August, 2022; originally announced August 2022.

  46. arXiv:2207.08426  [pdf, other

    cs.GT cs.LG cs.MA

    Fast Convergence of Optimistic Gradient Ascent in Network Zero-Sum Extensive Form Games

    Authors: Georgios Piliouras, Lillian Ratliff, Ryann Sim, Stratis Skoulakis

    Abstract: The study of learning in games has thus far focused primarily on normal form games. In contrast, our understanding of learning in extensive form games (EFGs) and particularly in EFGs with many agents lags far behind, despite them being closer in nature to many real world applications. We consider the natural class of Network Zero-Sum Extensive Form Games, which combines the global zero-sum propert… ▽ More

    Submitted 18 July, 2022; originally announced July 2022.

    Comments: To appear in SAGT 2022

  47. arXiv:2206.04160  [pdf, other

    cs.GT cs.LG math.DS

    Alternating Mirror Descent for Constrained Min-Max Games

    Authors: Andre Wibisono, Molei Tao, Georgios Piliouras

    Abstract: In this paper we study two-player bilinear zero-sum games with constrained strategy spaces. An instance of natural occurrences of such constraints is when mixed strategies are used, which correspond to a probability simplex constraint. We propose and analyze the alternating mirror descent algorithm, in which each player takes turns to take action following the mirror descent algorithm for constrai… ▽ More

    Submitted 8 June, 2022; originally announced June 2022.

  48. arXiv:2203.14129  [pdf, other

    cs.GT cs.LG econ.TH math.DS

    Nash, Conley, and Computation: Impossibility and Incompleteness in Game Dynamics

    Authors: Jason Milionis, Christos Papadimitriou, Georgios Piliouras, Kelly Spendlove

    Abstract: Under what conditions do the behaviors of players, who play a game repeatedly, converge to a Nash equilibrium? If one assumes that the players' behavior is a discrete-time or continuous-time rule whereby the current mixed strategy profile is mapped to the next, this becomes a problem in the theory of dynamical systems. We apply this theory, and in particular the concepts of chain recurrence, attra… ▽ More

    Submitted 26 March, 2022; originally announced March 2022.

    Comments: 25 pages

  49. arXiv:2203.11973  [pdf, other

    cs.LG math.OC stat.ML

    Scalable Deep Reinforcement Learning Algorithms for Mean Field Games

    Authors: Mathieu Laurière, Sarah Perrin, Sertan Girgin, Paul Muller, Ayush Jain, Theophile Cabannes, Georgios Piliouras, Julien Pérolat, Romuald Élie, Olivier Pietquin, Matthieu Geist

    Abstract: Mean Field Games (MFGs) have been introduced to efficiently approximate games with very large populations of strategic agents. Recently, the question of learning equilibria in MFGs has gained momentum, particularly using model-free reinforcement learning (RL) methods. One limiting factor to further scale up using RL is that existing algorithms to solve MFGs require the mixing of approximated quant… ▽ More

    Submitted 17 June, 2022; v1 submitted 22 March, 2022; originally announced March 2022.

  50. arXiv:2202.11871  [pdf, other

    cs.GT cs.LG math.DS

    No-Regret Learning in Games is Turing Complete

    Authors: Gabriel P. Andrade, Rafael Frongillo, Georgios Piliouras

    Abstract: Games are natural models for multi-agent machine learning settings, such as generative adversarial networks (GANs). The desirable outcomes from algorithmic interactions in these games are encoded as game theoretic equilibrium concepts, e.g. Nash and coarse correlated equilibria. As directly computing an equilibrium is typically impractical, one often aims to design learning algorithms that iterati… ▽ More

    Submitted 23 February, 2022; originally announced February 2022.

    Comments: 18 pages, 1 figure