Showing 1–50 of 136 results for author: Dorfler, F

Split-as-a-Pro: behavioral control via operator splitting and alternating projections

Authors: Yu Tang, Carlo Cenedese, Alessio Rimoldi, Florian Dórfler, John Lygeros, Alberto Padoan

Abstract: The paper introduces Split-as-a-Pro, a control framework that integrates behavioral systems theory, operator splitting methods, and alternating projection algorithms. The framework reduces dynamic optimization problems - arising in both control and estimation - to efficient projection computations. Split-as-a-Pro builds on a non-parametric formulation that exploits system structure to separate dyn… ▽ More The paper introduces Split-as-a-Pro, a control framework that integrates behavioral systems theory, operator splitting methods, and alternating projection algorithms. The framework reduces dynamic optimization problems - arising in both control and estimation - to efficient projection computations. Split-as-a-Pro builds on a non-parametric formulation that exploits system structure to separate dynamic constraints imposed by individual subsystems from external ones, such as interconnection constraints and input/output constraints. This enables the use of arbitrary system representations, as long as the associated projection is efficiently computable, thereby enhancing scalability and compatibility with gray-box modeling. We demonstrate the effectiveness of Split-as-a-Pro by developing a distributed algorithm for solving finite-horizon linear quadratic control problems and illustrate its use in predictive control. Our numerical case studies show that algorithms obtained using Split-as-a-Pro significantly outperform their centralized counterparts in runtime and scalability across various standard graph topologies, while seamlessly leveraging both model-based and data-driven system representations. △ Less

Submitted 25 May, 2025; originally announced May 2025.

Policy Gradient Adaptive Control for the LQR: Indirect and Direct Approaches

Authors: Feiran Zhao, Alessandro Chiuso, Florian Dörfler

Abstract: Motivated by recent advances of reinforcement learning and direct data-driven control, we propose policy gradient adaptive control (PGAC) for the linear quadratic regulator (LQR), which uses online closed-loop data to improve the control policy while maintaining stability. Our method adaptively updates the policy in feedback by descending the gradient of the LQR cost and is categorized as indirect… ▽ More Motivated by recent advances of reinforcement learning and direct data-driven control, we propose policy gradient adaptive control (PGAC) for the linear quadratic regulator (LQR), which uses online closed-loop data to improve the control policy while maintaining stability. Our method adaptively updates the policy in feedback by descending the gradient of the LQR cost and is categorized as indirect, when gradients are computed via an estimated model, versus direct, when gradients are derived from data using sample covariance parameterization. Beyond the vanilla gradient, we also showcase the merits of the natural gradient and Gauss-Newton methods for the policy update. Notably, natural gradient descent bridges the indirect and direct PGAC, and the Gauss-Newton method of the indirect PGAC leads to an adaptive version of the celebrated Hewer's algorithm. To account for the uncertainty from noise, we propose a regularization method for both indirect and direct PGAC. For all the considered PGAC approaches, we show closed-loop stability and convergence of the policy to the optimal LQR gain. Simulations validate our theoretical findings and demonstrate the robustness and computational efficiency of PGAC. △ Less

Submitted 6 May, 2025; originally announced May 2025.

Gaussian behaviors: representations and data-driven control

Authors: András Sasfi, Ivan Markovsky, Alberto Padoan, Florian Dörfler

Abstract: We propose a modeling framework for stochastic systems based on Gaussian processes. Finite-length trajectories of the system are modeled as random vectors from a Gaussian distribution, which we call a Gaussian behavior. The proposed model naturally quantifies the uncertainty in the trajectories, yet it is simple enough to allow for tractable formulations. We relate the proposed model to existing d… ▽ More We propose a modeling framework for stochastic systems based on Gaussian processes. Finite-length trajectories of the system are modeled as random vectors from a Gaussian distribution, which we call a Gaussian behavior. The proposed model naturally quantifies the uncertainty in the trajectories, yet it is simple enough to allow for tractable formulations. We relate the proposed model to existing descriptions of dynamical systems including deterministic and stochastic behaviors, and linear time-invariant (LTI) state-space models with Gaussian process and measurement noise. Gaussian behaviors can be estimated directly from observed data as the empirical sample covariance under the assumption that the measured trajectories are from independent experiments. The distribution of future outputs conditioned on inputs and past outputs provides a predictive model that can be incorporated in predictive control frameworks. We show that subspace predictive control (SPC) is a certainty-equivalence control formulation with the estimated Gaussian behavior. Furthermore, the regularized data-enabled predictive control (DeePC) method is shown to be a distributionally optimistic formulation that optimistically accounts for uncertainty in the Gaussian behavior. To mitigate the excessive optimism of DeePC, we propose a novel distributionally robust control formulation, and provide a convex reformulation allowing for efficient implementation. △ Less

Submitted 22 April, 2025; originally announced April 2025.

Reactive power flow optimization in AC drive systems

Authors: Sanjay Chandrasekaran, Catalin Arghir, Pieder Joerg, Florian Doerfler, Silvia Mastellone

Abstract: This paper explores a limit avoidance approach in the case of input (modulation) and output (current) constraints with the aim of enhancing system availability of AC drives. Drawing on the observation that, in a certain range of reactive power, there exists a trade-off between current and modulation magnitude, we exploit this freedom and define a constrained optimization problem. We propose two ap… ▽ More This paper explores a limit avoidance approach in the case of input (modulation) and output (current) constraints with the aim of enhancing system availability of AC drives. Drawing on the observation that, in a certain range of reactive power, there exists a trade-off between current and modulation magnitude, we exploit this freedom and define a constrained optimization problem. We propose two approaches, one in the form of an activation-function which drives the reactive power set-point towards safety, and an approach which uses online feedback optimization to set the reactive power dynamically. Both methods compromise reactive power tracking accuracy for increased system robustness. Through a high fidelity simulation, we compare the benefits of the two methods, highlighting their effectiveness in industrial applications. △ Less

Submitted 14 April, 2025; originally announced April 2025.

Comments: Submitted to the Conference on Decision and Control, 2025

Robust Feedback Optimization with Model Uncertainty: A Regularization Approach

Authors: Winnie Chan, Zhiyu He, Keith Moffat, Saverio Bolognani, Michael Muehlebach, Florian Dörfler

Abstract: Feedback optimization optimizes the steady state of a dynamical system by implementing optimization iterations in closed loop with the plant. It relies on online measurements and limited model information, namely, the input-output sensitivity. In practice, various issues including inaccurate modeling, lack of observation, or changing conditions can lead to sensitivity mismatches, causing closed-lo… ▽ More Feedback optimization optimizes the steady state of a dynamical system by implementing optimization iterations in closed loop with the plant. It relies on online measurements and limited model information, namely, the input-output sensitivity. In practice, various issues including inaccurate modeling, lack of observation, or changing conditions can lead to sensitivity mismatches, causing closed-loop sub-optimality or even instability. To handle such uncertainties, we pursue robust feedback optimization, where we optimize the closed-loop performance against all possible sensitivities lying in specific uncertainty sets. We provide tractable reformulations for the corresponding min-max problems via regularizations and characterize the online closed-loop performance through the tracking error in case of time-varying optimal solutions. Simulations on a distribution grid illustrate the effectiveness of our robust feedback optimization controller in addressing sensitivity mismatches in a non-stationary environment. △ Less

Submitted 31 March, 2025; originally announced March 2025.

Mixed Small Gain and Phase Theorem: A new view using Scale Relative Graphs

Authors: Eder Baron-Prada, Adolfo Anta, Alberto Padoan, Florian Dörfler

Abstract: We introduce a novel approach to feedback stability analysis for linear time-invariant (LTI) systems, overcoming the limitations of the sectoriality assumption in the small phase theorem. While phase analysis for single-input single-output (SISO) systems is well-established, multi-input multi-output (MIMO) systems lack a comprehensive phase analysis until recent advances introduced with the small-… ▽ More We introduce a novel approach to feedback stability analysis for linear time-invariant (LTI) systems, overcoming the limitations of the sectoriality assumption in the small phase theorem. While phase analysis for single-input single-output (SISO) systems is well-established, multi-input multi-output (MIMO) systems lack a comprehensive phase analysis until recent advances introduced with the small-phase theorem. A limitation of the small-phase theorem is the sectorial condition, which states that an operator's eigenvalues must lie within a specified angle sector of the complex plane. We propose a framework based on Scaled Relative Graphs (SRGs) to remove this assumption. We derive two main results: a graphical set-based stability condition using SRGs and a small-phase theorem with no sectorial assumption. These results broaden the scope of phase analysis and feedback stability for MIMO systems. △ Less

Submitted 17 March, 2025; originally announced March 2025.

Comments: To appear in ECC 2025

Decision-Dependent Stochastic Optimization: The Role of Distribution Dynamics

Authors: Zhiyu He, Saverio Bolognani, Florian Dörfler, Michael Muehlebach

Abstract: Distribution shifts have long been regarded as troublesome external forces that a decision-maker should either counteract or conform to. An intriguing feedback phenomenon termed decision dependence arises when the deployed decision affects the environment and alters the data-generating distribution. In the realm of performative prediction, this is encoded by distribution maps parameterized by deci… ▽ More Distribution shifts have long been regarded as troublesome external forces that a decision-maker should either counteract or conform to. An intriguing feedback phenomenon termed decision dependence arises when the deployed decision affects the environment and alters the data-generating distribution. In the realm of performative prediction, this is encoded by distribution maps parameterized by decisions due to strategic behaviors. In contrast, we formalize an endogenous distribution shift as a feedback process featuring nonlinear dynamics that couple the evolving distribution with the decision. Stochastic optimization in this dynamic regime provides a fertile ground to examine the various roles played by dynamics in the composite problem structure. To this end, we develop an online algorithm that achieves optimal decision-making by both adapting to and shaping the dynamic distribution. Throughout the paper, we adopt a distributional perspective and demonstrate how this view facilitates characterizations of distribution dynamics and the optimality and generalization performance of the proposed algorithm. We showcase the theoretical results in an opinion dynamics context, where an opportunistic party maximizes the affinity of a dynamic polarized population, and in a recommender system scenario, featuring performance optimization with discrete distributions in the probability simplex. △ Less

Submitted 10 March, 2025; originally announced March 2025.

Regularization for Covariance Parameterization of Direct Data-Driven LQR Control

Authors: Feiran Zhao, Alessandro Chiuso, Florian Dörfler

Abstract: As the benchmark of data-driven control methods, the linear quadratic regulator (LQR) problem has gained significant attention. A growing trend is direct LQR design, which finds the optimal LQR gain directly from raw data and bypassing system identification. To achieve this, our previous work develops a direct LQR formulation parameterized by sample covariance. In this paper, we propose a regulari… ▽ More As the benchmark of data-driven control methods, the linear quadratic regulator (LQR) problem has gained significant attention. A growing trend is direct LQR design, which finds the optimal LQR gain directly from raw data and bypassing system identification. To achieve this, our previous work develops a direct LQR formulation parameterized by sample covariance. In this paper, we propose a regularization method for the covariance-parameterized LQR. We show that the regularizer accounts for the uncertainty in both the steady-state covariance matrix corresponding to closed-loop stability, and the LQR cost function corresponding to averaged control performance. With a positive or negative coefficient, the regularizer can be interpreted as promoting either exploitation or exploration, which are well-known trade-offs in reinforcement learning. In simulations, we observe that our covariance-parameterized LQR with regularization can significantly outperform the certainty-equivalence LQR in terms of both the optimality gap and the robust closed-loop stability. △ Less

Submitted 4 March, 2025; originally announced March 2025.

Comments: Submitted to C-LSS and CDC

An Adaptive Data-Enabled Policy Optimization Approach for Autonomous Bicycle Control

Authors: Niklas Persson, Feiran Zhao, Mojtaba Kaheni, Florian Dörfler, Alessandro V. Papadopoulos

Abstract: This paper presents a unified control framework that integrates a Feedback Linearization (FL) controller in the inner loop with an adaptive Data-Enabled Policy Optimization (DeePO) controller in the outer loop to balance an autonomous bicycle. While the FL controller stabilizes and partially linearizes the inherently unstable and nonlinear system, its performance is compromised by unmodeled dynami… ▽ More This paper presents a unified control framework that integrates a Feedback Linearization (FL) controller in the inner loop with an adaptive Data-Enabled Policy Optimization (DeePO) controller in the outer loop to balance an autonomous bicycle. While the FL controller stabilizes and partially linearizes the inherently unstable and nonlinear system, its performance is compromised by unmodeled dynamics and time-varying characteristics. To overcome these limitations, the DeePO controller is introduced to enhance adaptability and robustness. The initial control policy of DeePO is obtained from a finite set of offline, persistently exciting input and state data. To improve stability and compensate for system nonlinearities and disturbances, a robustness-promoting regularizer refines the initial policy, while the adaptive section of the DeePO framework is enhanced with a forgetting factor to improve adaptation to time-varying dynamics. The proposed DeePO+FL approach is evaluated through simulations and real-world experiments on an instrumented autonomous bicycle. Results demonstrate its superiority over the FL-only approach, achieving more precise tracking of the reference lean angle and lean rate. △ Less

Submitted 19 February, 2025; originally announced February 2025.

Optimizing Social Network Interventions via Hypergradient-Based Recommender System Design

Authors: Marino Kühne, Panagiotis D. Grontas, Giulia De Pasquale, Giuseppe Belgioioso, Florian Dörfler, John Lygeros

Abstract: Although social networks have expanded the range of ideas and information accessible to users, they are also criticized for amplifying the polarization of user opinions. Given the inherent complexity of these phenomena, existing approaches to counteract these effects typically rely on handcrafted algorithms and heuristics. We propose an elegant solution: we act on the network weights that model us… ▽ More Although social networks have expanded the range of ideas and information accessible to users, they are also criticized for amplifying the polarization of user opinions. Given the inherent complexity of these phenomena, existing approaches to counteract these effects typically rely on handcrafted algorithms and heuristics. We propose an elegant solution: we act on the network weights that model user interactions on social networks (e.g., frequency of communication), to optimize a performance metric (e.g., polarization reduction), while users' opinions follow the classical Friedkin-Johnsen model. Our formulation gives rise to a challenging large-scale optimization problem with non-convex constraints, for which we develop a gradient-based algorithm. Our scheme is simple, scalable, and versatile, as it can readily integrate different, potentially non-convex, objectives. We demonstrate its merit by: (i) rapidly solving complex social network intervention problems with 3 million variables based on the Reddit and DBLP datasets; (ii) significantly outperforming competing approaches in terms of both computation time and disagreement reduction. △ Less

Submitted 18 February, 2025; originally announced February 2025.

A Stability Condition for Online Feedback Optimization without Timescale Separation

Authors: Mattia Bianchi, Florian Dörfler

Abstract: Online Feedback Optimization (OFO) is a control approach to drive a dynamical plant to an optimal steady state. By interconnecting optimization algorithms with real-time plant measurements, OFO provides all the benefits of feedback control, yet without requiring exact knowledge of plant dynamics for computing a setpoint. On the downside, existing stability guarantees for OFO require the controller… ▽ More Online Feedback Optimization (OFO) is a control approach to drive a dynamical plant to an optimal steady state. By interconnecting optimization algorithms with real-time plant measurements, OFO provides all the benefits of feedback control, yet without requiring exact knowledge of plant dynamics for computing a setpoint. On the downside, existing stability guarantees for OFO require the controller to evolve on a sufficiently slower timescale than the plant, possibly affecting transient performance and responsiveness to disturbances. In this paper, we prove that, under suitable conditions, OFO ensures stability without any timescale separation. In particular, the condition we propose is independent of the time constant of the plant, hence it is scaling-invariant. Our analysis leverages a composite Lyapunov function, which is the $\max$ of plant-related and controller-related components. We corroborate our theoretical results with numerical examples. △ Less

Submitted 14 December, 2024; originally announced December 2024.

Subspace tracking for online system identification

Authors: András Sasfi, Alberto Padoan, Ivan Markovsky, Florian Dörfler

Abstract: This paper introduces an online approach for identifying time-varying subspaces defined by linear dynamical systems, leveraging optimization on the Grassmannian manifold leading to the Grassmannian Recursive Algorithm for Tracking (GREAT) method. The approach of representing linear systems by non-parametric subspace models has received significant interest in the field of data-driven control recen… ▽ More This paper introduces an online approach for identifying time-varying subspaces defined by linear dynamical systems, leveraging optimization on the Grassmannian manifold leading to the Grassmannian Recursive Algorithm for Tracking (GREAT) method. The approach of representing linear systems by non-parametric subspace models has received significant interest in the field of data-driven control recently. We view subspaces as points on the Grassmannian manifold, and therefore, tracking is achieved by performing optimization on the manifold. At each time step, a single measurement from the current subspace corrupted by a bounded error is available. The subspace estimate is updated online using Grassmannian gradient descent on a cost function incorporating a window of the most recent data. Under suitable assumptions on the signal-to-noise ratio of the online data and the subspace's rate of change, we establish theoretical guarantees for the resulting algorithm. More specifically, we prove an exponential convergence rate and provide a consistent uncertainty quantification of the estimates in terms of an upper bound on their distance to the true subspace. The applicability of the proposed algorithm is demonstrated by means of numerical examples, and it is shown to compare favorably with competing parametric system identification methods. △ Less

Submitted 12 December, 2024; originally announced December 2024.

Comments: Submitted to IEEE Transactions on Automatic Control

DeePC-Hunt: Data-enabled Predictive Control Hyperparameter Tuning via Differentiable Optimization

Authors: Michael Cummins, Alberto Padoan, Keith Moffat, Florian Dorfler, John Lygeros

Abstract: This paper introduces Data-enabled Predictive Control Hyperparameter Tuning via Differentiable Optimization (DeePC-Hunt), a backpropagation-based method for automatic hyperparameter tuning of the DeePC algorithm. The necessity for such a method arises from the importance of hyperparameter selection to achieve satisfactory closed-loop DeePC performance. The standard methods for hyperparameter selec… ▽ More This paper introduces Data-enabled Predictive Control Hyperparameter Tuning via Differentiable Optimization (DeePC-Hunt), a backpropagation-based method for automatic hyperparameter tuning of the DeePC algorithm. The necessity for such a method arises from the importance of hyperparameter selection to achieve satisfactory closed-loop DeePC performance. The standard methods for hyperparameter selection are to either optimize the open-loop performance, or use manual guess-and-check. Optimizing the open-loop performance can result in unacceptable closed-loop behavior, while manual guess-and-check can pose safety challenges. DeePC-Hunt provides an alternative method for hyperparameter tuning which uses an approximate model of the system dynamics and backpropagation to directly optimize hyperparameters for the closed-loop DeePC performance. Numerical simulations demonstrate the effectiveness of DeePC in combination with DeePC-Hunt in a complex stabilization task for a nonlinear system and its superiority over model-based control strategies in terms of robustness to model misspecifications. △ Less

Submitted 29 May, 2025; v1 submitted 9 December, 2024; originally announced December 2024.

Comments: L4DC 2025

On analysis of open optimization algorithms

Authors: Jaap Eising, Florian Dörfler

Abstract: We develop analysis results for optimization algorithms that are open, that is, with inputs and outputs. Such algorithms arise for instance, when analyzing the effect of noise or disturbance on an algorithm, or when an algorithm is part of control loop without timescale separation. To be precise, we consider an incremental small gain problem to analyze robustness. Moreover, we investigate the beha… ▽ More We develop analysis results for optimization algorithms that are open, that is, with inputs and outputs. Such algorithms arise for instance, when analyzing the effect of noise or disturbance on an algorithm, or when an algorithm is part of control loop without timescale separation. To be precise, we consider an incremental small gain problem to analyze robustness. Moreover, we investigate the behaviors of the closed loop between incrementally dissipative nonlinear plants and optimization algorithms. The framework we develop is built upon the theories of incremental dissipativity and monotone operators, and yields tests in the form of linear matrix inequalities. △ Less

Submitted 27 November, 2024; originally announced November 2024.

Data-Driven Structured Robust Control of Linear Systems

Authors: Jared Miller, Jaap Eising, Florian Dörfler, Roy S. Smith

Abstract: Static structured control refers to the task of designing a state-feedback controller such that the control gain satisfies a subspace constraint. Structured control has applications in control of communication-inhibited dynamical systems, such as systems in networked environments. This work performs $H_2$-suboptimal regulation under a common structured state-feedback controller for a class of data… ▽ More Static structured control refers to the task of designing a state-feedback controller such that the control gain satisfies a subspace constraint. Structured control has applications in control of communication-inhibited dynamical systems, such as systems in networked environments. This work performs $H_2$-suboptimal regulation under a common structured state-feedback controller for a class of data-consistent plants. The certification of $H_2$-performance is attained through a combination of standard $H_2$ LMIs, convex sufficient conditions for structured control, and a matrix S-lemma for set-membership. The resulting convex optimization problems are linear matrix inequalities whose size scales independently of the number of data samples collected. Data-driven structured $H_2$-regulation control is demonstrated on example systems. △ Less

Submitted 18 November, 2024; originally announced November 2024.

Comments: 7 pages

Direct Adaptive Control of Grid-Connected Power Converters via Output-Feedback Data-Enabled Policy Optimization

Authors: Feiran Zhao, Ruohan Leng, Linbin Huang, Huanhai Xin, Keyou You, Florian Dörfler

Abstract: Power electronic converters are becoming the main components of modern power systems due to the increasing integration of renewable energy sources. However, power converters may become unstable when interacting with the complex and time-varying power grid. In this paper, we propose an adaptive data-driven control method to stabilize power converters by using only online input-output data. Our cont… ▽ More Power electronic converters are becoming the main components of modern power systems due to the increasing integration of renewable energy sources. However, power converters may become unstable when interacting with the complex and time-varying power grid. In this paper, we propose an adaptive data-driven control method to stabilize power converters by using only online input-output data. Our contributions are threefold. First, we reformulate the output-feedback control problem as a state-feedback linear quadratic regulator (LQR) problem with a controllable non-minimal state, which can be constructed from past input-output signals. Second, we propose a data-enabled policy optimization (DeePO) method for this non-minimal realization to achieve efficient output-feedback adaptive control. Third, we use high-fidelity simulations to verify that the output-feedback DeePO can effectively stabilize grid-connected power converters and quickly adapt to the changes in the power grid. △ Less

Submitted 8 April, 2025; v1 submitted 6 November, 2024; originally announced November 2024.

Optimality of Linear Policies for Distributionally Robust Linear Quadratic Gaussian Regulator with Stationary Distributions

Authors: Nicolas Lanzetti, Antonio Terpin, Florian Dörfler

Abstract: We prove that output-feedback linear policies remain optimal for solving the Linear Quadratic Gaussian regulation problem in the face of worst-case process and measurement noise distributions when these are independent, stationary, and known to be within a radius (in the Wasserstein sense) to some reference zero-mean Gaussian noise distributions. Additionally, we establish the existence of a Nash… ▽ More We prove that output-feedback linear policies remain optimal for solving the Linear Quadratic Gaussian regulation problem in the face of worst-case process and measurement noise distributions when these are independent, stationary, and known to be within a radius (in the Wasserstein sense) to some reference zero-mean Gaussian noise distributions. Additionally, we establish the existence of a Nash equilibrium of the zero-sum game between a control engineer, who minimizes control cost, and a fictitious adversary, who chooses the noise distributions that maximize this cost. For general (possibly non-Gaussian) reference noise distributions, we establish a quasi closed-form solution for the worst-case distributions against linear policies. Our work provides a less conservative alternative compared to recent work in distributionally robust control. △ Less

Submitted 22 April, 2025; v1 submitted 30 October, 2024; originally announced October 2024.

Comments: Accepted for presentation at, and publication in the proceedings of, the 2025 European Control Conference

Contractivity and linear convergence in bilinear saddle-point problems: An operator-theoretic approach

Authors: Colin Dirren, Mattia Bianchi, Panagiotis D. Grontas, John Lygeros, Florian Dörfler

Abstract: We study the convex-concave bilinear saddle-point problem $\min_x \max_y f(x) + y^\top Ax - g(y)$, where both, only one, or none of the functions $f$ and $g$ are strongly convex, and suitable rank conditions on the matrix $A$ hold. The solution of this problem is at the core of many machine learning tasks. By employing tools from monotone operator theory, we systematically prove the contractivity… ▽ More We study the convex-concave bilinear saddle-point problem $\min_x \max_y f(x) + y^\top Ax - g(y)$, where both, only one, or none of the functions $f$ and $g$ are strongly convex, and suitable rank conditions on the matrix $A$ hold. The solution of this problem is at the core of many machine learning tasks. By employing tools from monotone operator theory, we systematically prove the contractivity (in turn, the linear convergence) of several first-order primal-dual algorithms, including the Chambolle-Pock method. Our approach results in concise proofs, and it yields new convergence guarantees and tighter bounds compared to known results. △ Less

Submitted 21 April, 2025; v1 submitted 18 October, 2024; originally announced October 2024.

Comments: AISTATS 2025

Network-aware Recommender System via Online Feedback Optimization

Authors: Sanjay Chandrasekaran, Giulia De Pasquale, Giuseppe Belgioioso, Florian Dörfler

Abstract: Personalized content on social platforms can exacerbate negative phenomena such as polarization, partly due to the feedback interactions between recommendations and the users. In this paper, we present a control-theoretic recommender system that explicitly accounts for this feedback loop to mitigate polarization. Our approach extends online feedback optimization - a control paradigm for steady-sta… ▽ More Personalized content on social platforms can exacerbate negative phenomena such as polarization, partly due to the feedback interactions between recommendations and the users. In this paper, we present a control-theoretic recommender system that explicitly accounts for this feedback loop to mitigate polarization. Our approach extends online feedback optimization - a control paradigm for steady-state optimization of dynamical systems - to develop a recommender system that trades off users engagement and polarization reduction, while relying solely on online click data. We establish theoretical guarantees for optimality and stability of the proposed design and validate its effectiveness via numerical experiments with a user population governed by Friedkin-Johnsen dynamics. Our results show these "network-aware" recommendations can significantly reduce polarization while maintaining high levels of user engagement. △ Less

Submitted 26 September, 2024; v1 submitted 29 August, 2024; originally announced August 2024.

Fairness in Social Influence Maximization via Optimal Transport

Authors: Shubham Chowdhary, Giulia De Pasquale, Nicolas Lanzetti, Ana-Andreea Stoica, Florian Dorfler

Abstract: We study fairness in social influence maximization, whereby one seeks to select seeds that spread a given information throughout a network, ensuring balanced outreach among different communities (e.g. demographic groups). In the literature, fairness is often quantified in terms of the expected outreach within individual communities. In this paper, we demonstrate that such fairness metrics can be m… ▽ More We study fairness in social influence maximization, whereby one seeks to select seeds that spread a given information throughout a network, ensuring balanced outreach among different communities (e.g. demographic groups). In the literature, fairness is often quantified in terms of the expected outreach within individual communities. In this paper, we demonstrate that such fairness metrics can be misleading since they overlook the stochastic nature of information diffusion processes. When information diffusion occurs in a probabilistic manner, multiple outreach scenarios can occur. As such, outcomes such as ``In 50% of the cases, no one in group 1 gets the information, while everyone in group 2 does, and in the other 50%, it is the opposite'', which always results in largely unfair outcomes, are classified as fair by a variety of fairness metrics in the literature. We tackle this problem by designing a new fairness metric, mutual fairness, that captures variability in outreach through optimal transport theory. We propose a new seed-selection algorithm that optimizes both outreach and mutual fairness, and we show its efficacy on several real datasets. We find that our algorithm increases fairness with only a minor decrease (and at times, even an increase) in efficiency. △ Less

Submitted 30 January, 2025; v1 submitted 25 June, 2024; originally announced June 2024.

Variational Analysis in the Wasserstein Space

Authors: Nicolas Lanzetti, Antonio Terpin, Florian Dörfler

Abstract: We study optimization problems whereby the optimization variable is a probability measure. Since the probability space is not a vector space, many classical and powerful methods for optimization (e.g., gradients) are of little help. Thus, one typically resorts to the abstract machinery of infinite-dimensional analysis or other ad-hoc methodologies, not tailored to the probability space, which howe… ▽ More We study optimization problems whereby the optimization variable is a probability measure. Since the probability space is not a vector space, many classical and powerful methods for optimization (e.g., gradients) are of little help. Thus, one typically resorts to the abstract machinery of infinite-dimensional analysis or other ad-hoc methodologies, not tailored to the probability space, which however involve projections or rely on convexity-type assumptions. We believe instead that these problems call for a comprehensive methodological framework for calculus in probability spaces. In this work, we combine ideas from optimal transport, variational analysis, and Wasserstein gradient flows to equip the Wasserstein space (i.e., the space of probability measures endowed with the Wasserstein distance) with a variational structure, both by combining and extending existing results and introducing novel tools. Our theoretical analysis culminates in very general necessary optimality conditions for optimality. Notably, our conditions (i) resemble the rationales of Euclidean spaces, such as the Karush-Kuhn-Tucker and Lagrange conditions, (ii) are intuitive, informative, and easy to study, and (iii) yield closed-form solutions or can be used to design computationally attractive algorithms. We believe this framework lays the foundation for new algorithmic and theoretical advancements in the study of optimization problems in probability spaces, which we exemplify with numerous case studies and applications to machine learning, drug discovery, and distributionally robust optimization. △ Less

Submitted 15 June, 2024; originally announced June 2024.

Distributed Traffic Signal Control via Coordinated Maximum Pressure-plus-Penalty

Authors: Vinzenz Tütsch, Zhiyu He, Florian Dörfler, Kenan Zhang

Abstract: This paper develops an adaptive traffic control policy inspired by Maximum Pressure (MP) while imposing coordination across intersections. The proposed Coordinated Maximum Pressure-plus-Penalty (CMPP) control policy features a local objective for each intersection that consists of the total pressure within the neighborhood and a penalty accounting for the queue capacities and continuous green time… ▽ More This paper develops an adaptive traffic control policy inspired by Maximum Pressure (MP) while imposing coordination across intersections. The proposed Coordinated Maximum Pressure-plus-Penalty (CMPP) control policy features a local objective for each intersection that consists of the total pressure within the neighborhood and a penalty accounting for the queue capacities and continuous green time for certain movements. The corresponding control task is reformulated as a distributed optimization problem and solved via two customized algorithms: one based on the alternating direction method of multipliers (ADMM) and the other follows a greedy heuristic augmented with a majority vote. CMPP not only provides a theoretical guarantee of queuing network stability but also outperforms several benchmark controllers in simulations on a large-scale real traffic network with lower average travel and waiting time per vehicle, as well as less network congestion. Furthermore, CPMM with the greedy algorithm enjoys comparable computational efficiency as fully decentralized controllers without significantly compromising the control performance, which highlights its great potential for real-world deployment. △ Less

Submitted 30 April, 2024; originally announced April 2024.

Constrained multi-cluster game: Distributed Nash equilibrium seeking over directed graphs

Authors: Duong Thuy Anh Nguyen, Mattia Bianchi, Florian Dörfler, Duong Tung Nguyen, Angelia Nedić

Abstract: Motivated by the complex dynamics of cooperative and competitive interactions within networked agent systems, multi-cluster games provide a framework for modeling the interconnected goals of self-interested clusters of agents. For this setup, the existing literature lacks comprehensive gradient-based solutions that simultaneously consider constraint sets and directed communication networks, both o… ▽ More Motivated by the complex dynamics of cooperative and competitive interactions within networked agent systems, multi-cluster games provide a framework for modeling the interconnected goals of self-interested clusters of agents. For this setup, the existing literature lacks comprehensive gradient-based solutions that simultaneously consider constraint sets and directed communication networks, both of which are crucial for many practical applications. To address this gap, this paper proposes a distributed Nash equilibrium seeking algorithm that integrates consensus-based methods and gradient-tracking techniques, where inter-cluster and intra-cluster communications only use row- and column-stochastic weight matrices, respectively. To handle constraints, we introduce an averaging procedure, which can effectively address the complications associated with projections. In turn, we can show linear convergence of our algorithm, focusing on the contraction property of the optimality gap. We demonstrate the efficacy of the proposed algorithm through a microgrid energy management application. △ Less

Submitted 22 April, 2024; originally announced April 2024.

Gray-Box Nonlinear Feedback Optimization

Authors: Zhiyu He, Saverio Bolognani, Michael Muehlebach, Florian Dörfler

Abstract: Feedback optimization enables autonomous optimality seeking of a dynamical system through its closed-loop interconnection with iterative optimization algorithms. Among various iteration structures, model-based approaches require the input-output sensitivity of the system to construct gradients, whereas model-free approaches bypass this need by estimating gradients from real-time evaluations of the… ▽ More Feedback optimization enables autonomous optimality seeking of a dynamical system through its closed-loop interconnection with iterative optimization algorithms. Among various iteration structures, model-based approaches require the input-output sensitivity of the system to construct gradients, whereas model-free approaches bypass this need by estimating gradients from real-time evaluations of the objective. These approaches own complementary benefits in sample efficiency and accuracy against model mismatch, i.e., errors of sensitivities. To achieve the best of both worlds, we propose gray-box feedback optimization controllers, featuring systematic incorporation of approximate sensitivities into model-free updates via adaptive convex combination. We quantify conditions on the accuracy of the sensitivities that render the gray-box approach preferable. We elucidate how the closed-loop performance is determined by the number of iterations, the problem dimension, and the cumulative effect of inaccurate sensitivities. The proposed controller contributes to a balanced closed-loop behavior, which retains provable sample efficiency and optimality guarantees for nonconvex problems. We further develop a running gray-box controller to handle constrained time-varying problems with changing objectives and steady-state maps. △ Less

Submitted 5 April, 2024; originally announced April 2024.

Mitigating Transient Bullwhip Effects Under Imperfect Demand Forecasts

Authors: Sarah H. Q. Li, Florian Dörfler

Abstract: Motivated by how forecast errors exacerbate order fluctuations in supply chains, we leverage robust feedback controller synthesis to characterize, compute, and minimize the worst-case order fluctuation experienced by an individual supply chain vendor. Assuming bounded forecast errors and demand fluctuations, we model forecast error and demand fluctuations as inputs to linear inventory dynamics, an… ▽ More Motivated by how forecast errors exacerbate order fluctuations in supply chains, we leverage robust feedback controller synthesis to characterize, compute, and minimize the worst-case order fluctuation experienced by an individual supply chain vendor. Assuming bounded forecast errors and demand fluctuations, we model forecast error and demand fluctuations as inputs to linear inventory dynamics, and use the $\ell_\infty$ gain to define a transient Bullwhip measure. In contrast to the existing Bullwhip measure, the transient Bullwhip measure explicitly depends on the forecast error. This enables us to separately quantify the transient Bullwhip measure's sensitivity to forecast error and demand fluctuations. To compute the controller that minimizes the worst-case peak gain, we formulate an optimization problem with bilinear matrix inequalities and show that it is equivalent to minimizing a quasi-convex function on a bounded domain. We simulate our model for vendors with non-zero perishable rates and order backlogging rates, and prove that the transient Bullwhip measure can be bounded by a monotonic quasi-convex function whose dependency on the product backlog rate and perishing rate is verified in simulation. △ Less

Submitted 12 September, 2024; v1 submitted 1 April, 2024; originally announced April 2024.

Comments: 7 pages, 4 figures

Online Feedback Optimization over Networks: A Distributed Model-free Approach

Authors: Wenbin Wang, Zhiyu He, Giuseppe Belgioioso, Saverio Bolognani, Florian Dörfler

Abstract: Online feedback optimization (OFO) enables optimal steady-state operations of a physical system by employing an iterative optimization algorithm as a dynamic feedback controller. When the plant consists of several interconnected sub-systems, centralized implementations become impractical due to the heavy computational burden and the need to pre-compute system-wide sensitivities, which may not be e… ▽ More Online feedback optimization (OFO) enables optimal steady-state operations of a physical system by employing an iterative optimization algorithm as a dynamic feedback controller. When the plant consists of several interconnected sub-systems, centralized implementations become impractical due to the heavy computational burden and the need to pre-compute system-wide sensitivities, which may not be easily accessible in practice. Motivated by these challenges, we develop a fully distributed model-free OFO controller, featuring consensus-based tracking of the global objective value and local iterative (projected) updates that use stochastic gradient estimates. We characterize how the closed-loop performance depends on the size of the network, the number of iterations, and the level of accuracy of consensus. Numerical simulations on a voltage control problem in a direct current power grid corroborate the theoretical findings. △ Less

Submitted 12 September, 2024; v1 submitted 28 March, 2024; originally announced March 2024.

A Coupled Optimization Framework for Correlated Equilibria in Normal-Form Game

Authors: Sarah H. Q. Li, Yue Yu, Florian Dörfler, John Lygeros

Abstract: In competitive multi-player interactions, simultaneous optimality is a key requirement for establishing strategic equilibria. This property is explicit when the game-theoretic equilibrium is the simultaneously optimal solution of coupled optimization problems. However, no such optimization problems exist for the correlated equilibrium, a strategic equilibrium where the players can correlate their… ▽ More In competitive multi-player interactions, simultaneous optimality is a key requirement for establishing strategic equilibria. This property is explicit when the game-theoretic equilibrium is the simultaneously optimal solution of coupled optimization problems. However, no such optimization problems exist for the correlated equilibrium, a strategic equilibrium where the players can correlate their actions. We address the lack of a coupled optimization framework for the correlated equilibrium by introducing an {unnormalized game} -- an extension of normal-form games in which the player strategies are lifted to unnormalized measures over the joint actions. We show that the set of fully mixed generalized Nash equilibria of this unnormalized game is a subset of the correlated equilibrium of the normal-form game. Furthermore, we introduce an entropy regularization to the unnormalized game and prove that the entropy-regularized generalized Nash equilibrium is a sub-optimal correlated equilibrium of the normal form game where the degree of sub-optimality depends on the magnitude of regularization. We prove that the entropy-regularized unnormalized game has a closed-form solution, and empirically verify its computational efficacy at approximating the correlated equilibrium of normal-form games. △ Less

Submitted 3 April, 2024; v1 submitted 24 March, 2024; originally announced March 2024.

Comments: 8 pages, 2 figures

Optimal control of continuous-time symmetric systems with unknown dynamics and noisy measurements

Authors: Hamed Taghavian, Florian Dorfler, Mikael Johansson

Abstract: An iterative learning algorithm is presented for continuous-time linear-quadratic optimal control problems where the system is externally symmetric with unknown dynamics. Both finite-horizon and infinite-horizon problems are considered. It is shown that the proposed algorithm is globally convergent to the optimal solution and has some advantages over adaptive dynamic programming, including being u… ▽ More An iterative learning algorithm is presented for continuous-time linear-quadratic optimal control problems where the system is externally symmetric with unknown dynamics. Both finite-horizon and infinite-horizon problems are considered. It is shown that the proposed algorithm is globally convergent to the optimal solution and has some advantages over adaptive dynamic programming, including being unbiased under noisy measurements and having a relatively low computational burden. Numerical experiments show the effectiveness of the results. △ Less

Submitted 20 March, 2024; originally announced March 2024.

Data-Enabled Policy Optimization for Direct Adaptive Learning of the LQR

Authors: Feiran Zhao, Florian Dörfler, Alessandro Chiuso, Keyou You

Abstract: Direct data-driven design methods for the linear quadratic regulator (LQR) mainly use offline or episodic data batches, and their online adaptation has been acknowledged as an open problem. In this paper, we propose a direct adaptive method to learn the LQR from online closed-loop data. First, we propose a new policy parameterization based on the sample covariance to formulate a direct data-driven… ▽ More Direct data-driven design methods for the linear quadratic regulator (LQR) mainly use offline or episodic data batches, and their online adaptation has been acknowledged as an open problem. In this paper, we propose a direct adaptive method to learn the LQR from online closed-loop data. First, we propose a new policy parameterization based on the sample covariance to formulate a direct data-driven LQR problem, which is shown to be equivalent to the certainty-equivalence LQR with optimal non-asymptotic guarantees. Second, we design a novel data-enabled policy optimization (DeePO) method to directly update the policy, where the gradient is explicitly computed using only a batch of persistently exciting (PE) data. Third, we establish its global convergence via a projected gradient dominance property. Importantly, we efficiently use DeePO to adaptively learn the LQR by performing only one-step projected gradient descent per sample of the closed-loop system, which also leads to an explicit recursive update of the policy. Under PE inputs and for bounded noise, we show that the average regret of the LQR cost is upper-bounded by two terms signifying a sublinear decrease in time $\mathcal{O}(1/\sqrt{T})$ plus a bias scaling inversely with signal-to-noise ratio (SNR), which are independent of the noise statistics. Finally, we perform simulations to validate the theoretical results and demonstrate the computational and sample efficiency of our method. △ Less

Submitted 4 October, 2024; v1 submitted 26 January, 2024; originally announced January 2024.

Comments: Submitted to IEEE TAC

Towards a Systems Theory of Algorithms

Authors: Florian Dörfler, Zhiyu He, Giuseppe Belgioioso, Saverio Bolognani, John Lygeros, Michael Muehlebach

Abstract: Traditionally, numerical algorithms are seen as isolated pieces of code confined to an {\em in silico} existence. However, this perspective is not appropriate for many modern computational approaches in control, learning, or optimization, wherein {\em in vivo} algorithms interact with their environment. Examples of such {\em open algorithms} include various real-time optimization-based control str… ▽ More Traditionally, numerical algorithms are seen as isolated pieces of code confined to an {\em in silico} existence. However, this perspective is not appropriate for many modern computational approaches in control, learning, or optimization, wherein {\em in vivo} algorithms interact with their environment. Examples of such {\em open algorithms} include various real-time optimization-based control strategies, reinforcement learning, decision-making architectures, online optimization, and many more. Further, even {\em closed} algorithms in learning or optimization are increasingly abstracted in block diagrams with interacting dynamic modules and pipelines. In this opinion paper, we state our vision on a to-be-cultivated {\em systems theory of algorithms} and argue in favor of viewing algorithms as open dynamical systems interacting with other algorithms, physical systems, humans, or databases. Remarkably, the manifold tools developed under the umbrella of systems theory are well suited for addressing a range of challenges in the algorithmic domain. We survey various instances where the principles of algorithmic systems theory are being developed and outline pertinent modeling, analysis, and design challenges. △ Less

Submitted 30 April, 2024; v1 submitted 25 January, 2024; originally announced January 2024.

Distributionally Robust Infinite-horizon Control: from a pool of samples to the design of dependable controllers

Authors: Jean-Sébastien Brouillon, Andrea Martin, John Lygeros, Florian Dörfler, Giancarlo Ferrari Trecate

Abstract: We study control of constrained linear systems with only partial statistical information about the uncertainty affecting the system dynamics and the sensor measurements. Specifically, given a finite collection of disturbance realizations drawn from a generic distribution, we consider the problem of designing a stabilizing control policy with provable safety and performance guarantees despite the m… ▽ More We study control of constrained linear systems with only partial statistical information about the uncertainty affecting the system dynamics and the sensor measurements. Specifically, given a finite collection of disturbance realizations drawn from a generic distribution, we consider the problem of designing a stabilizing control policy with provable safety and performance guarantees despite the mismatch between the empirical and true distributions. We capture this discrepancy using Wasserstein ambiguity sets, and we formulate a distributionally robust (DR) optimal control problem, which provides guarantees on the expected cost, safety, and stability of the system. To solve this problem, we first present new results for DR optimization of quadratic objectives using convex programming, showing that strong duality holds under mild conditions. Then, by combining our results with the system-level parametrization of linear feedback policies, we show that the design problem can be reduced to a semidefinite program. We present numerical simulations to validate the effectiveness of our approach and to highlight the value of empirical distributions for control design. △ Less

Submitted 11 July, 2024; v1 submitted 12 December, 2023; originally announced December 2023.

Urban traffic congestion control: a DeePC change

Authors: Alessio Rimoldi, Carlo Cenedese, Alberto Padoan, Florian Dörfler, John Lygeros

Abstract: Urban traffic congestion remains a pressing challenge in our rapidly expanding cities, despite the abundance of available data and the efforts of policymakers. By leveraging behavioral system theory and data-driven control, this paper exploits the DeePC algorithm in the context of urban traffic control performed via dynamic traffic lights. To validate our approach, we consider a high-fidelity case… ▽ More Urban traffic congestion remains a pressing challenge in our rapidly expanding cities, despite the abundance of available data and the efforts of policymakers. By leveraging behavioral system theory and data-driven control, this paper exploits the DeePC algorithm in the context of urban traffic control performed via dynamic traffic lights. To validate our approach, we consider a high-fidelity case study using the state-of-the-art simulation software package Simulation of Urban MObility (SUMO). Preliminary results indicate that DeePC outperforms existing approaches across various key metrics, including travel time and CO$_2$ emissions, demonstrating its potential for effective traffic management △ Less

Submitted 16 November, 2023; originally announced November 2023.

Comments: This paper has been submitted to IEEE ECC24

Decentralized Feedback Optimization via Sensitivity Decoupling: Stability and Sub-optimality

Authors: Wenbin Wang, Zhiyu He, Giuseppe Belgioioso, Saverio Bolognani, Florian Dörfler

Abstract: Online feedback optimization is a controller design paradigm for optimizing the steady-state behavior of a dynamical system. It employs an optimization algorithm as a dynamic feedback controller and utilizes real-time measurements to bypass knowing exact plant dynamics and disturbances. Different from existing centralized settings, we present a fully decentralized feedback optimization controller… ▽ More Online feedback optimization is a controller design paradigm for optimizing the steady-state behavior of a dynamical system. It employs an optimization algorithm as a dynamic feedback controller and utilizes real-time measurements to bypass knowing exact plant dynamics and disturbances. Different from existing centralized settings, we present a fully decentralized feedback optimization controller for networked systems to lift the communication burden and improve scalability. We approximate the overall input-output sensitivity matrix through its diagonal elements, which capture local model information. For the closed-loop behavior, we characterize the stability and bound the sub-optimality due to decentralization. We prove that the proposed decentralized controller yields solutions that correspond to the Nash equilibria of a non-cooperative game. △ Less

Submitted 28 March, 2024; v1 submitted 15 November, 2023; originally announced November 2023.

Efficient Exploration in Continuous-time Model-based Reinforcement Learning

Authors: Lenart Treven, Jonas Hübotter, Bhavya Sukhija, Florian Dörfler, Andreas Krause

Abstract: Reinforcement learning algorithms typically consider discrete-time dynamics, even though the underlying systems are often continuous in time. In this paper, we introduce a model-based reinforcement learning algorithm that represents continuous-time dynamics using nonlinear ordinary differential equations (ODEs). We capture epistemic uncertainty using well-calibrated probabilistic models, and use t… ▽ More Reinforcement learning algorithms typically consider discrete-time dynamics, even though the underlying systems are often continuous in time. In this paper, we introduce a model-based reinforcement learning algorithm that represents continuous-time dynamics using nonlinear ordinary differential equations (ODEs). We capture epistemic uncertainty using well-calibrated probabilistic models, and use the optimistic principle for exploration. Our regret bounds surface the importance of the measurement selection strategy(MSS), since in continuous time we not only must decide how to explore, but also when to observe the underlying system. Our analysis demonstrates that the regret is sublinear when modeling ODEs with Gaussian Processes (GP) for common choices of MSS, such as equidistant sampling. Additionally, we propose an adaptive, data-dependent, practical MSS that, when combined with GP dynamics, also achieves sublinear regret with significantly fewer samples. We showcase the benefits of continuous-time modeling over its discrete-time counterpart, as well as our proposed adaptive MSS over standard baselines, on several applications. △ Less

Submitted 30 October, 2023; originally announced October 2023.

Tutorial on Congestion Control in Multi-Area Transmission Grids via Online Feedback Equilibrium Seeking

Authors: Giuseppe Belgioioso, Saverio Bolognani, Giulia Pejrani, Florian Dörfler

Abstract: Online feedback optimization (OFO) is an emerging control methodology for real-time optimal steady-state control of complex dynamical systems. This tutorial focuses on the application of OFO for the autonomous operation of large-scale transmission grids, with a specific goal of minimizing renewable generation curtailment and losses while satisfying voltage and current limits. When this control met… ▽ More Online feedback optimization (OFO) is an emerging control methodology for real-time optimal steady-state control of complex dynamical systems. This tutorial focuses on the application of OFO for the autonomous operation of large-scale transmission grids, with a specific goal of minimizing renewable generation curtailment and losses while satisfying voltage and current limits. When this control methodology is applied to multi-area transmission grids, where each area independently manages its congestion while being dynamically interconnected with the rest of the grid, a non-cooperative game arises. In this context, OFO must be interpreted as an online feedback equilibrium seeking (FES) scheme. Our analysis incorporates technical tools from game theory and monotone operator theory to evaluate the stability and performance of multi-area grid operation. Through numerical simulations, we illustrate the key challenge of this non-cooperative setting: on the one hand, independent multi-area decisions are suboptimal compared to a centralized control scheme; on the other hand, some areas are heavily penalized by the centralized decision, which may discourage participation in the coordination mechanism. △ Less

Submitted 24 October, 2023; originally announced October 2023.

Data-driven representations of conical, convex, and affine behaviors

Authors: Alberto Padoan, Florian Dörfler, John Lygeros

Abstract: The paper studies conical, convex, and affine models in the framework of behavioral systems theory. We investigate basic properties of such behaviors and address the problem of constructing models from measured data. We prove that closed, shift-invariant, conical, convex, and affine models have the intersection property, thereby enabling the definition of most powerful unfalsified models based on… ▽ More The paper studies conical, convex, and affine models in the framework of behavioral systems theory. We investigate basic properties of such behaviors and address the problem of constructing models from measured data. We prove that closed, shift-invariant, conical, convex, and affine models have the intersection property, thereby enabling the definition of most powerful unfalsified models based on infinite-horizon measurements. We then provide necessary and sufficient conditions for representing conical, convex, and affine finite-horizon behaviors using raw data matrices, expressing persistence of excitation requirements in terms of non-negative rank conditions. The applicability of our results is demonstrated by a numerical example arising in population ecology. △ Less

Submitted 23 October, 2023; originally announced October 2023.

Controller implementability: a data-driven approach

Authors: Alberto Padoan, Jeremy Coulson, Florian Dörfler

Abstract: We study the controller implementability problem, which seeks to determine if a controller can make the closed-loop behavior of a given plant match that of a desired reference behavior. We establish necessary and sufficient conditions for controller implementability which only rely on raw data. Subsequently, we consider the problem of constructing controllers directly from data. By leveraging the… ▽ More We study the controller implementability problem, which seeks to determine if a controller can make the closed-loop behavior of a given plant match that of a desired reference behavior. We establish necessary and sufficient conditions for controller implementability which only rely on raw data. Subsequently, we consider the problem of constructing controllers directly from data. By leveraging the concept of canonical controller, we provide a formula to directly construct controllers that implement plant-compatible reference behaviors using measurements of both reference and plant behaviors. △ Less

Submitted 23 October, 2023; originally announced October 2023.

Physics-Informed Graph Neural Network for Dynamic Reconfiguration of Power Systems

Authors: Jules Authier, Rabab Haider, Anuradha Annaswamy, Florian Dorfler

Abstract: To maintain a reliable grid we need fast decision-making algorithms for complex problems like Dynamic Reconfiguration (DyR). DyR optimizes distribution grid switch settings in real-time to minimize grid losses and dispatches resources to supply loads with available generation. DyR is a mixed-integer problem and can be computationally intractable to solve for large grids and at fast timescales. We… ▽ More To maintain a reliable grid we need fast decision-making algorithms for complex problems like Dynamic Reconfiguration (DyR). DyR optimizes distribution grid switch settings in real-time to minimize grid losses and dispatches resources to supply loads with available generation. DyR is a mixed-integer problem and can be computationally intractable to solve for large grids and at fast timescales. We propose GraPhyR, a Physics-Informed Graph Neural Network (GNNs) framework tailored for DyR. We incorporate essential operational and connectivity constraints directly within the GNN framework and train it end-to-end. Our results show that GraPhyR is able to learn to optimize the DyR task. △ Less

Submitted 2 April, 2024; v1 submitted 1 October, 2023; originally announced October 2023.

Comments: 8 pages, 5 figures, 2 tables. To appear at PSCC 2024

Joint Oscillation Damping and Inertia Provision Service for Converter-Interfaced Generation

Authors: Cheng Feng, Linbin Huang, Xiuqiang He, Yi Wang, Florian Dörfler, Chongqing Kang

Abstract: Power systems dominated by converter-interfaced distributed energy resources (DERs) typically exhibit weaker damping capabilities and lower inertia, compromising system stability. Although individual DER controllers are evolving to provide superior oscillation damping capabilities and inertia supports, there is a lack of network-wide coordinated management measures for multiple DERs, potentially l… ▽ More Power systems dominated by converter-interfaced distributed energy resources (DERs) typically exhibit weaker damping capabilities and lower inertia, compromising system stability. Although individual DER controllers are evolving to provide superior oscillation damping capabilities and inertia supports, there is a lack of network-wide coordinated management measures for multiple DERs, potentially leading to unexpected instability and cost-effectiveness problems. To address this gap, this paper introduces a hybrid oscillation damping and inertia management strategy for multiple DERs, considering network coupling effects, and seeks to encourage DERs to provide enhanced damping and inertia with appropriate economic incentives. We first formulate an optimization problem to tune and allocate damping and inertia coefficients for DERs, minimizing associated power and energy costs while ensuring hard constraints for system frequency stability and small-signal stability. The problem is built upon a novel convex parametric formulation that integrates oscillation mode location and frequency trajectory requirements, equipped with a theoretical guarantee, and eliminating the need for iterative tuning and computation burdens. Furthermore, to increase the willingness of DERs to cooperate, we further design appropriate economic incentives to compensate for DERs' costs based on the proposed cost minimization problem, and assess its impact on system cost-efficiency. Numerical tests highlight the effectiveness of the proposed method in promoting system stability and offer insights into potential economic benefits. △ Less

Submitted 18 April, 2025; v1 submitted 3 September, 2023; originally announced September 2023.

Comments: Accepted by IEEE TPWRS. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses

Strategic Interactions in Multi-modal Mobility Systems: A Game-Theoretic Perspective

Authors: Gioele Zardini, Nicolas Lanzetti, Giuseppe Belgioioso, Christian Hartnik, Saverio Bolognani, Florian Dörfler, Emilio Frazzoli

Abstract: The evolution of existing transportation systems,mainly driven by urbanization and increased availability of mobility options, such as private, profit-maximizing ride-hailing companies, calls for tools to reason about their design and regulation. To study this complex socio-technical problem, one needs to account for the strategic interactions of the heterogeneous stakeholders involved in the mobi… ▽ More The evolution of existing transportation systems,mainly driven by urbanization and increased availability of mobility options, such as private, profit-maximizing ride-hailing companies, calls for tools to reason about their design and regulation. To study this complex socio-technical problem, one needs to account for the strategic interactions of the heterogeneous stakeholders involved in the mobility ecosystem and analyze how they influence the system. In this paper, we focus on the interactions between citizens who compete for the limited resources of a mobility system to complete their desired trip. Specifically, we present a game-theoretic framework for multi-modal mobility systems, where citizens, characterized by heterogeneous preferences, have access to various mobility options and seek individually-optimal decisions. We study the arising game and prove the existence of an equilibrium, which can be efficiently computed via a convex optimization problem. Through both an analytical and a numerical case study for the classic scenario of Sioux Falls, USA, we illustrate the capabilities of our model and perform sensitivity analyses. Importantly, we show how to embed our framework into a "larger" game among stakeholders of the mobility ecosystem (e.g., municipality, Mobility Service Providers, and citizens), effectively giving rise to tools to inform strategic interventions and policy-making in the mobility ecosystem. △ Less

Submitted 9 August, 2023; originally announced August 2023.

Comments: 8 pages, 5 figures, to appear in the proceedings of the 2023 IEEE 26th International Conference on Intelligent Transportation Systems

Grid-Forming Hybrid Angle Control: Behavior, Stability, Variants and Verification

Authors: Ali Tayyebi, Denis Vettoretti, Adolfo Anta, Florian Dörfler

Abstract: This work explores the stability, behavior, variants, and a controller-hardware-in-the-loop (C-HiL) verification of the recently proposed grid-forming (GFM) hybrid angle control (HAC). We revisit the foundation of GFM HAC, and highlight its behavioral properties in relation to the conventional synchronous machine (SM). Next, we introduce the required complementary controls to be combined with the… ▽ More This work explores the stability, behavior, variants, and a controller-hardware-in-the-loop (C-HiL) verification of the recently proposed grid-forming (GFM) hybrid angle control (HAC). We revisit the foundation of GFM HAC, and highlight its behavioral properties in relation to the conventional synchronous machine (SM). Next, we introduce the required complementary controls to be combined with the HAC to realize a GFM behavior. The characterization of the analytical operating point and nonlinear energy-based stability analysis of a grid-connected converter under the HAC is presented. Further, we consider various output filter configurations and derive an approximation for the original control proposal. Moreover, we provide details on the integration of GFM HAC into a complex converter control architecture and introduce several variants of the standard HAC. Finally, the performance of GFM HAC is verified by several test scenarios in a C-HiL setup to test its behavior against real-world effect such as noise and delays. △ Less

Submitted 18 July, 2023; originally announced July 2023.

SOS Construction of Compatible Control Lyapunov and Barrier Functions

Authors: Michael Schneeberger, Florian Dörfler, Silvia Mastellone

Abstract: We propose a novel approach to certify closed-loop stability and safety of a constrained polynomial system based on the combination of Control Lyapunov Functions (CLFs) and Control Barrier Functions (CBFs). For polynomial systems that are affine in the control input, both classes of functions can be constructed via Sum Of Squares (SOS) programming. Using two versions of the Positivstellensatz we d… ▽ More We propose a novel approach to certify closed-loop stability and safety of a constrained polynomial system based on the combination of Control Lyapunov Functions (CLFs) and Control Barrier Functions (CBFs). For polynomial systems that are affine in the control input, both classes of functions can be constructed via Sum Of Squares (SOS) programming. Using two versions of the Positivstellensatz we derive an SOS formulation seeking a rational controller that - if feasible - results in compatible CLF and multiple CBFs. △ Less

Submitted 2 May, 2023; originally announced May 2023.

Wasserstein Tube MPC with Exact Uncertainty Propagation

Authors: Liviu Aolaritei, Marta Fochesato, John Lygeros, Florian Dörfler

Abstract: We study model predictive control (MPC) problems for stochastic LTI systems, where the noise distribution is unknown, compactly supported, and only observable through a limited number of i.i.d. noise samples. Building upon recent results in the literature, which show that distributional uncertainty can be efficiently captured within a Wasserstein ambiguity set, and that such ambiguity sets propaga… ▽ More We study model predictive control (MPC) problems for stochastic LTI systems, where the noise distribution is unknown, compactly supported, and only observable through a limited number of i.i.d. noise samples. Building upon recent results in the literature, which show that distributional uncertainty can be efficiently captured within a Wasserstein ambiguity set, and that such ambiguity sets propagate exactly through the system dynamics, we start by formulating a novel Wasserstein Tube MPC (WT-MPC) problem, with distributionally robust CVaR constraints. We then show that the WT-MPC problem: (1) is a direct generalization of the (deterministic) Robust Tube MPC (RT-MPC) to the stochastic setting; (2) through a scalar parameter, it interpolates between the data-driven formulation based on sample average approximation and the RT-MPC formulation, allowing us to optimally trade between safety and performance; (3) admits a tractable convex reformulation; and (4) is recursively feasible. We conclude the paper with a numerical comparison of WT-MPC and RT-MPC. △ Less

Submitted 24 April, 2023; originally announced April 2023.

Designing Optimal Personalized Incentive for Traffic Routing using BIG Hype algorithm

Authors: Panagiotis D. Grontas, Carlo Cenedese, Marta Fochesato, Giuseppe Belgioioso, John Lygeros, Florian Dörfler

Abstract: We study the problem of optimally routing plug-in electric and conventional fuel vehicles on a city level. In our model, commuters selfishly aim to minimize a local cost that combines travel time, from a fixed origin to a desired destination, and the monetary cost of using city facilities, parking or service stations. The traffic authority can influence the commuters' preferred routing choice by m… ▽ More We study the problem of optimally routing plug-in electric and conventional fuel vehicles on a city level. In our model, commuters selfishly aim to minimize a local cost that combines travel time, from a fixed origin to a desired destination, and the monetary cost of using city facilities, parking or service stations. The traffic authority can influence the commuters' preferred routing choice by means of personalized discounts on parking tickets and on the energy price at service stations. We formalize the problem of designing these monetary incentives optimally as a large-scale bilevel game, where constraints arise at both levels due to the finite capacities of city facilities and incentives budget. Then, we develop an efficient decentralized solution scheme with convergence guarantees based on BIG Hype, a recently-proposed hypergradient-based algorithm for hierarchical games. Finally, we validate our model via numerical simulations over the Anaheim's network, and show that the proposed approach produces sensible results in terms of traffic decongestion and it is able to solve in minutes problems with more than 48000 variables and 110000 constraints. △ Less

Submitted 24 April, 2023; originally announced April 2023.

Comments: Submitted to IEEE CDC23

Capture, Propagate, and Control Distributional Uncertainty

Authors: Liviu Aolaritei, Nicolas Lanzetti, Florian Dörfler

Abstract: We study stochastic dynamical systems in settings where only partial statistical information about the noise is available, e.g., in the form of a limited number of noise realizations. Such systems are particularly challenging to analyze and control, primarily due to an absence of a distributional uncertainty model which: (1) is expressive enough to capture practically relevant scenarios; (2) can b… ▽ More We study stochastic dynamical systems in settings where only partial statistical information about the noise is available, e.g., in the form of a limited number of noise realizations. Such systems are particularly challenging to analyze and control, primarily due to an absence of a distributional uncertainty model which: (1) is expressive enough to capture practically relevant scenarios; (2) can be easily propagated through system maps; (3) is invariant under propagation; and (4) allows for computationally tractable control actions. In this paper, we propose to model distributional uncertainty via Optimal Transport ambiguity sets and show that such modeling choice satisfies all of the above requirements. We then specialize our results to stochastic LTI systems, and start by showing that the distributional uncertainty can be efficiently captured, with high probability, within an Optimal Transport ambiguity set on the space of noise trajectories. Then, we show that such ambiguity sets propagate exactly through the system dynamics, giving rise to stochastic tubes that contain, with high probability, all trajectories of the stochastic system. Finally, we show that the control task is very interpretable, unveiling an interesting decomposition between the roles of the feedforward and the feedback control terms. Our results are actionable and successfully applied in stochastic reachability analysis and in trajectory planning under distributional uncertainty. △ Less

Submitted 5 April, 2023; originally announced April 2023.

Comments: arXiv admin note: text overlap with arXiv:2205.00343

Data-enabled Policy Optimization for the Linear Quadratic Regulator

Authors: Feiran Zhao, Florian Dörfler, Keyou You

Abstract: Policy optimization (PO), an essential approach of reinforcement learning for a broad range of system classes, requires significantly more system data than indirect (identification-followed-by-control) methods or behavioral-based direct methods even in the simplest linear quadratic regulator (LQR) problem. In this paper, we take an initial step towards bridging this gap by proposing the data-enabl… ▽ More Policy optimization (PO), an essential approach of reinforcement learning for a broad range of system classes, requires significantly more system data than indirect (identification-followed-by-control) methods or behavioral-based direct methods even in the simplest linear quadratic regulator (LQR) problem. In this paper, we take an initial step towards bridging this gap by proposing the data-enabled policy optimization (DeePO) method, which requires only a finite number of sufficiently exciting data to iteratively solve the LQR problem via PO. Based on a data-driven closed-loop parameterization, we are able to directly compute the policy gradient from a batch of persistently exciting data. Next, we show that the nonconvex PO problem satisfies a projected gradient dominance property by relating it to an equivalent convex program, leading to the global convergence of DeePO. Moreover, we apply regularization methods to enhance certainty-equivalence and robustness of the resulting controller and show an implicit regularization property. Finally, we perform simulations to validate our results. △ Less

Submitted 15 September, 2023; v1 submitted 31 March, 2023; originally announced March 2023.

Comments: Accepted in IEEE CDC 2023

Nash Equilibria, Regularization and Computation in Optimal Transport-Based Distributionally Robust Optimization

Authors: Soroosh Shafiee, Liviu Aolaritei, Florian Dörfler, Daniel Kuhn

Abstract: We study optimal transport-based distributionally robust optimization problems where a fictitious adversary, often envisioned as nature, can choose the distribution of the uncertain problem parameters by reshaping a prescribed reference distribution at a finite transportation cost. In this framework, we show that robustification is intimately related to various forms of variation and Lipschitz reg… ▽ More We study optimal transport-based distributionally robust optimization problems where a fictitious adversary, often envisioned as nature, can choose the distribution of the uncertain problem parameters by reshaping a prescribed reference distribution at a finite transportation cost. In this framework, we show that robustification is intimately related to various forms of variation and Lipschitz regularization even if the transportation cost function fails to be (some power of) a metric. We also derive conditions for the existence and the computability of a Nash equilibrium between the decision-maker and nature, and we demonstrate numerically that nature's Nash strategy can be viewed as a distribution that is supported on remarkably deceptive adversarial samples. Finally, we identify practically relevant classes of optimal transport-based distributionally robust optimization problems that can be addressed with efficient gradient descent algorithms even if the loss function or the transportation cost function are nonconvex (but not both at the same time). △ Less

Submitted 1 June, 2025; v1 submitted 7 March, 2023; originally announced March 2023.

BIG Hype: Best Intervention in Games via Distributed Hypergradient Descent

Authors: Panagiotis D. Grontas, Giuseppe Belgioioso, Carlo Cenedese, Marta Fochesato, John Lygeros, Florian Dörfler

Abstract: Hierarchical decision making problems, such as bilevel programs and Stackelberg games, are attracting increasing interest in both the engineering and machine learning communities. Yet, existing solution methods lack either convergence guarantees or computational efficiency, due to the absence of smoothness and convexity. In this work, we bridge this gap by designing a first-order hypergradient-bas… ▽ More Hierarchical decision making problems, such as bilevel programs and Stackelberg games, are attracting increasing interest in both the engineering and machine learning communities. Yet, existing solution methods lack either convergence guarantees or computational efficiency, due to the absence of smoothness and convexity. In this work, we bridge this gap by designing a first-order hypergradient-based algorithm for Stackelberg games and mathematically establishing its convergence using tools from nonsmooth analysis. To evaluate the \textit{hypergradient}, namely, the gradient of the upper-level objective, we develop an online scheme that simultaneously computes the lower-level equilibrium and its Jacobian. Crucially, this scheme exploits and preserves the original hierarchical and distributed structure of the problem, which renders it scalable and privacy-preserving. We numerically verify the computational efficiency and scalability of our algorithm on a large-scale hierarchical demand-response model. △ Less

Submitted 28 March, 2024; v1 submitted 2 March, 2023; originally announced March 2023.

Dynamic Programming in Probability Spaces via Optimal Transport

Authors: Antonio Terpin, Nicolas Lanzetti, Florian Dörfler

Abstract: We study discrete-time finite-horizon optimal control problems in probability spaces, whereby the state of the system is a probability measure. We show that, in many instances, the solution of dynamic programming in probability spaces results from two ingredients: (i) the solution of dynamic programming in the "ground space" (i.e., the space on which the probability measures live) and (ii) the sol… ▽ More We study discrete-time finite-horizon optimal control problems in probability spaces, whereby the state of the system is a probability measure. We show that, in many instances, the solution of dynamic programming in probability spaces results from two ingredients: (i) the solution of dynamic programming in the "ground space" (i.e., the space on which the probability measures live) and (ii) the solution of an optimal transport problem. From a multi-agent control perspective, a separation principle holds: The "low-level control of the agents of the fleet" (how does one reach the destination?) and "fleet-level control" (who goes where?) are decoupled. △ Less

Submitted 8 April, 2024; v1 submitted 27 February, 2023; originally announced February 2023.

Journal ref: SIAM Journal on Control and Optimization, 62(2), 1183-1206 (2024)

Stability and Robustness of Distributed Suboptimal Model Predictive Control

Authors: Giuseppe Belgioioso, Dominic Liao-McPherson, Mathias Hudoba de Badyn, Nicolas Pelzmann, John Lygeros, Florian Dörfler

Abstract: In distributed model predictive control (MPC), the control input at each sampling time is computed by solving a large-scale optimal control problem (OCP) over a finite horizon using distributed algorithms. Typically, such algorithms require several (virtually, infinite) communication rounds between the subsystems to converge, which is a major drawback both computationally and from an energetic per… ▽ More In distributed model predictive control (MPC), the control input at each sampling time is computed by solving a large-scale optimal control problem (OCP) over a finite horizon using distributed algorithms. Typically, such algorithms require several (virtually, infinite) communication rounds between the subsystems to converge, which is a major drawback both computationally and from an energetic perspective (for wireless systems). Motivated by these challenges, we propose a suboptimal distributed MPC scheme in which the total communication burden is distributed also in time, by maintaining a running solution estimate for the large-scale OCP and updating it at each sampling time. We demonstrate that, under some regularity conditions, the resulting suboptimal MPC control law recovers the qualitative robust stability properties of optimal MPC, if the communication budget at each sampling time is large enough. △ Less

Submitted 27 March, 2023; v1 submitted 14 November, 2022; originally announced November 2022.