Showing 1–19 of 19 results for author: di Bernardo, M

Controlling Complex Systems

Authors: Marco Coraggio, Davide Salzano, Mario di Bernardo

Abstract: This chapter provides a comprehensive overview of controlling collective behavior in complex systems comprising large ensembles of interacting dynamical agents. Building upon traditional control theory's foundation in individual systems, we introduce tools designed to address the unique challenges of coordinating networks that exhibit emergent phenomena, including consensus, synchronization, and p… ▽ More This chapter provides a comprehensive overview of controlling collective behavior in complex systems comprising large ensembles of interacting dynamical agents. Building upon traditional control theory's foundation in individual systems, we introduce tools designed to address the unique challenges of coordinating networks that exhibit emergent phenomena, including consensus, synchronization, and pattern formation. We analyze how local agent interactions generate macroscopic behaviors and investigate the fundamental role of network topology in determining system dynamics. Inspired by natural systems, we emphasize control strategies that achieve global coordination through localized interventions while considering practical implementation challenges. The chapter concludes by presenting novel frameworks for managing very large agent ensembles and leveraging interacting networks for control purposes. △ Less

Submitted 10 April, 2025; originally announced April 2025.

GoRINNs: Godunov-Riemann Informed Neural Networks for Learning Hyperbolic Conservation Laws

Authors: Dimitrios G. Patsatzis, Mario di Bernardo, Lucia Russo, Constantinos Siettos

Abstract: We present GoRINNs: numerical analysis-informed (shallow) neural networks for the solution of inverse problems of non-linear systems of conservation laws. GoRINNs is a hybrid/blended machine learning scheme based on high-resolution Godunov schemes for the solution of the Riemann problem in hyperbolic Partial Differential Equations (PDEs). In contrast to other existing machine learning methods that… ▽ More We present GoRINNs: numerical analysis-informed (shallow) neural networks for the solution of inverse problems of non-linear systems of conservation laws. GoRINNs is a hybrid/blended machine learning scheme based on high-resolution Godunov schemes for the solution of the Riemann problem in hyperbolic Partial Differential Equations (PDEs). In contrast to other existing machine learning methods that learn the numerical fluxes or just parameters of conservative Finite Volume methods, relying on deep neural networks (that may lead to poor approximations due to the computational complexity involved in their training), GoRINNs learn the closures of the conservation laws per se based on "intelligently" numerical-assisted shallow neural networks. Due to their structure, in particular, GoRINNs provide explainable, conservative schemes, that solve the inverse problem for hyperbolic PDEs, on the basis of approximate Riemann solvers that satisfy the Rankine-Hugoniot condition. The performance of GoRINNs is assessed via four benchmark problems, namely the Burgers', the Shallow Water, the Lighthill-Whitham-Richards and the Payne-Whitham traffic flow models. The solution profiles of these PDEs exhibit shock waves, rarefactions and/or contact discontinuities at finite times. We demonstrate that GoRINNs provide a very high accuracy both in the smooth and discontinuous regions. △ Less

Submitted 10 November, 2024; v1 submitted 29 October, 2024; originally announced October 2024.

Comments: 29 pages, 6 figures

MSC Class: 65M08; 65M32; 35L65; 68T07

CT-DQN: Control-Tutored Deep Reinforcement Learning

Authors: Francesco De Lellis, Marco Coraggio, Giovanni Russo, Mirco Musolesi, Mario di Bernardo

Abstract: One of the major challenges in Deep Reinforcement Learning for control is the need for extensive training to learn the policy. Motivated by this, we present the design of the Control-Tutored Deep Q-Networks (CT-DQN) algorithm, a Deep Reinforcement Learning algorithm that leverages a control tutor, i.e., an exogenous control law, to reduce learning time. The tutor can be designed using an approxima… ▽ More One of the major challenges in Deep Reinforcement Learning for control is the need for extensive training to learn the policy. Motivated by this, we present the design of the Control-Tutored Deep Q-Networks (CT-DQN) algorithm, a Deep Reinforcement Learning algorithm that leverages a control tutor, i.e., an exogenous control law, to reduce learning time. The tutor can be designed using an approximate model of the system, without any assumption about the knowledge of the system's dynamics. There is no expectation that it will be able to achieve the control objective if used stand-alone. During learning, the tutor occasionally suggests an action, thus partially guiding exploration. We validate our approach on three scenarios from OpenAI Gym: the inverted pendulum, lunar lander, and car racing. We demonstrate that CT-DQN is able to achieve better or equivalent data efficiency with respect to the classic function approximation solutions. △ Less

Submitted 2 December, 2022; originally announced December 2022.

Control-Tutored Reinforcement Learning: Towards the Integration of Data-Driven and Model-Based Control

Authors: F. De Lellis, M. Coraggio, G. Russo, M. Musolesi, M. di Bernardo

Abstract: We present an architecture where a feedback controller derived on an approximate model of the environment assists the learning process to enhance its data efficiency. This architecture, which we term as Control-Tutored Q-learning (CTQL), is presented in two alternative flavours. The former is based on defining the reward function so that a Boolean condition can be used to determine when the contro… ▽ More We present an architecture where a feedback controller derived on an approximate model of the environment assists the learning process to enhance its data efficiency. This architecture, which we term as Control-Tutored Q-learning (CTQL), is presented in two alternative flavours. The former is based on defining the reward function so that a Boolean condition can be used to determine when the control tutor policy is adopted, while the latter, termed as probabilistic CTQL (pCTQL), is instead based on executing calls to the tutor with a certain probability during learning. Both approaches are validated, and thoroughly benchmarked against Q-Learning, by considering the stabilization of an inverted pendulum as defined in OpenAI Gym as a representative problem. △ Less

Submitted 11 December, 2021; originally announced December 2021.

Intermittent non-pharmaceutical strategies to mitigate the COVID-19 epidemic in a network model of Italy via constrained optimization

Authors: Marco Coraggio, Shihao Xie, Francesco De Lellis, Giovanni Russo, Mario di Bernardo

Abstract: This paper is concerned with the design of intermittent non-pharmaceutical strategies to mitigate the spread of the COVID-19 epidemic exploiting network epidemiological models. Specifically, by studying a variational equation for the dynamics of the infected in a network model of the epidemic spread, we derive, using contractivity arguments, a condition that can be used to guarantee that, in epide… ▽ More This paper is concerned with the design of intermittent non-pharmaceutical strategies to mitigate the spread of the COVID-19 epidemic exploiting network epidemiological models. Specifically, by studying a variational equation for the dynamics of the infected in a network model of the epidemic spread, we derive, using contractivity arguments, a condition that can be used to guarantee that, in epidemiological terms, the effective reproduction number is less than unity. This condition has three advantages: (i) it is easily computable; (ii) it is directly related to the model parameters; (iii) it can be used to enforce a scalability condition that prohibits the amplification of disturbances within the network system. We then include satisfaction of such a condition as a constraint in a Model Predictive Control problem so as to mitigate (or suppress) the spread of the epidemic while minimizing the economic impact of the interventions. A data-driven model of Italy as a network of three macro-regions (North, Center, and South), whose parameters are identified from real data, is used to illustrate and evaluate the effectiveness of the proposed control strategy. △ Less

Submitted 4 October, 2021; v1 submitted 25 March, 2021; originally announced March 2021.

Comments: Accepted by Control and Decision Conference 2021

Tutoring Reinforcement Learning via Feedback Control

Authors: Francesco De Lellis, Giovanni Russo, Mario di Bernardo

Abstract: We introduce a control-tutored reinforcement learning (CTRL) algorithm. The idea is to enhance tabular learning algorithms by means of a control strategy with limited knowledge of the system model. By tutoring the learning process, the learning rate can be substantially reduced. We use the classical problem of stabilizing an inverted pendulum as a benchmark to numerically illustrate the advantages… ▽ More We introduce a control-tutored reinforcement learning (CTRL) algorithm. The idea is to enhance tabular learning algorithms by means of a control strategy with limited knowledge of the system model. By tutoring the learning process, the learning rate can be substantially reduced. We use the classical problem of stabilizing an inverted pendulum as a benchmark to numerically illustrate the advantages and disadvantages of the approach. △ Less

Submitted 12 December, 2020; originally announced December 2020.

Control-Tutored Reinforcement Learning

Authors: Francesco De Lellis, Fabrizia Auletta, Giovanni Russo, Piero De Lellis, Mario di Bernardo

Abstract: We introduce a control-tutored reinforcement learning (CTRL) algorithm. The idea is to enhance tabular learning algorithms so as to improve the exploration of the state-space, and substantially reduce learning times by leveraging some limited knowledge of the plant encoded into a tutoring model-based control strategy. We illustrate the benefits of our novel approach and its effectiveness by using… ▽ More We introduce a control-tutored reinforcement learning (CTRL) algorithm. The idea is to enhance tabular learning algorithms so as to improve the exploration of the state-space, and substantially reduce learning times by leveraging some limited knowledge of the plant encoded into a tutoring model-based control strategy. We illustrate the benefits of our novel approach and its effectiveness by using the problem of controlling one or more agents to herd and contain within a goal region a set of target free-roving agents in the plane. △ Less

Submitted 12 December, 2019; originally announced December 2019.

Reconstructing directed and weighted topologies of phase-locked oscillator networks

Authors: Francesco Alderisio, Gianfranco Fiore, Mario di Bernardo

Abstract: The formalism of complex networks is extensively employed to describe the dynamics of interacting agents in several applications. The features of the connections among the nodes in a network are not always provided beforehand, hence the problem of appropriately inferring them often arises. Here, we present a method to reconstruct directed and weighted topologies (REDRAW) of networks of heterogeneo… ▽ More The formalism of complex networks is extensively employed to describe the dynamics of interacting agents in several applications. The features of the connections among the nodes in a network are not always provided beforehand, hence the problem of appropriately inferring them often arises. Here, we present a method to reconstruct directed and weighted topologies (REDRAW) of networks of heterogeneous phase-locked nonlinear oscillators. We ultimately plan on using REDRAW to infer the interaction structure in human ensembles engaged in coordination tasks, and give insights into the overall behavior. △ Less

Submitted 10 November, 2016; originally announced November 2016.

Journal ref: Phys. Rev. E 95, 042302 (2017)

Interaction patterns and individual dynamics shape the way we move in synchrony

Authors: Francesco Alderisio, Gianfranco Fiore, Robin N Salesse, Benoit G Bardy, Mario di Bernardo

Abstract: An important open problem in Human Behaviour is to understand how coordination emerges in human ensembles. This problem has been seldom studied quantitatively in the existing literature, in contrast to situations involving dual interaction. Here we study motor coordination (or synchronisation) in a group of individuals where participants are asked to visually coordinate an oscillatory hand motion.… ▽ More An important open problem in Human Behaviour is to understand how coordination emerges in human ensembles. This problem has been seldom studied quantitatively in the existing literature, in contrast to situations involving dual interaction. Here we study motor coordination (or synchronisation) in a group of individuals where participants are asked to visually coordinate an oscillatory hand motion. We separately tested two groups of seven participants. We observed that the coordination level of the ensemble depends on group homogeneity, as well as on the pattern of visual couplings (who looked at whom). Despite the complexity of social interactions, we show that networks of coupled heterogeneous oscillators with different structures capture well the group dynamics. Our findings are relevant to any activity requiring the coordination of several people, as in music, sport or at work, and can be extended to account for other perceptual forms of interaction such as sound or feel. △ Less

Submitted 15 June, 2017; v1 submitted 7 July, 2016; originally announced July 2016.

Modeling Joint Improvisation between Human and Virtual Players in the Mirror Game

Authors: Chao Zhai, Francesco Alderisio, Piotr Slowinski, Krasimira Tsaneva-Atanasova, Mario di Bernardo

Abstract: Joint improvisation is observed to emerge spontaneously among humans performing joint action tasks, and has been associated with high levels of movement synchrony and enhanced sense of social bonding. Exploring the underlying cognitive and neural mechanisms behind the emergence of joint improvisation is an open research challenge. This paper investigates the emergence of jointly improvised movemen… ▽ More Joint improvisation is observed to emerge spontaneously among humans performing joint action tasks, and has been associated with high levels of movement synchrony and enhanced sense of social bonding. Exploring the underlying cognitive and neural mechanisms behind the emergence of joint improvisation is an open research challenge. This paper investigates the emergence of jointly improvised movements between two participants in the mirror game, a paradigmatic joint task example. A theoretical model based on observations and analysis of experimental data is proposed to capture the main features of their interaction. A set of experiments is carried out to test and validate the model ability to reproduce the experimental observations. Then, the model is used to drive a computer avatar able to improvise joint motion with a human participant in real time. Finally, a convergence analysis of the proposed model is carried out to confirm its ability to reproduce the emergence of joint movement between the participants. △ Less

Submitted 17 December, 2015; originally announced December 2015.

Design and Validation of a Virtual Player for Studying Interpersonal Coordination in the Mirror Game

Authors: Chao Zhai, Francesco Alderisio, Piotr Slowinski, Krasimira Tsaneva-Atanasova, Mario di Bernardo

Abstract: The mirror game has been recently proposed as a simple, yet powerful paradigm for studying interpersonal interactions. It has been suggested that a virtual partner able to play the game with human subjects can be an effective tool to affect the underlying neural processes needed to establish the necessary connections between the players, and also to provide new clinical interventions for the rehab… ▽ More The mirror game has been recently proposed as a simple, yet powerful paradigm for studying interpersonal interactions. It has been suggested that a virtual partner able to play the game with human subjects can be an effective tool to affect the underlying neural processes needed to establish the necessary connections between the players, and also to provide new clinical interventions for the rehabilitation of patients suffering from social disorders. Inspired by the motor processes of the central nervous system (CNS) and the musculoskeletal system in the human body, in this paper we develop a novel interactive cognitive architecture based on nonlinear control theory to drive a virtual player (VP) to play the mirror game with a human player (HP) in different configurations. Specifically, we consider two cases: the former where the VP acts as leader and the latter where it acts as follower. The crucial problem is to design a feedback control architecture capable of imitating and following or leading a human player (HP) in a joint action task. Movement of the end-effector of the VP is modeled by means of a feedback controlled Haken-Kelso-Bunz (HKB) oscillator, which is coupled with the observed motion of the HP measured in real time. To this aim, two types of control algorithms (adaptive control and optimal control) are used and implemented on the HKB model so that the VP can generate human-like motion while satisfying certain kinematic constraints. A proof of convergence of the control algorithms is presented in the paper together with an extensive numerical and experimental validation of their effectiveness. A comparison with other existing designs is also discussed, showing the flexibility and the advantages of our control-based approach. △ Less

Submitted 19 September, 2015; originally announced September 2015.

Entrainment and Synchronization in Heterogeneous Networks of Haken-Kelso-Bunz (HKB) Oscillators

Authors: Francesco Alderisio, Benoit G. Bardy, Mario di Bernardo

Abstract: In this paper we consider a heterogeneous network of Haken-Kelso-Bunz (HKB) nonlinear oscillators coupled through both linear and nonlinear interaction protocols. While some work exists on a system made up of only two nonlinearly coupled HKB oscillators as a model of human dynamics during interpersonal coordination tasks, the problem of considering a network of three or more HKBs has not been full… ▽ More In this paper we consider a heterogeneous network of Haken-Kelso-Bunz (HKB) nonlinear oscillators coupled through both linear and nonlinear interaction protocols. While some work exists on a system made up of only two nonlinearly coupled HKB oscillators as a model of human dynamics during interpersonal coordination tasks, the problem of considering a network of three or more HKBs has not been fully investigated. The aim of our work is to study convergence and synchronization in networks of HKB oscillators as a paradigm of coordination in multiplayer games. Convergence results are obtained under the assumption that the network is connected, simple and undirected. Analytical results are obtained to prove convergence when the oscillators are coupled diffusively. All theoretical results are illustrated via numerical examples. Finally, the effects of adding an external entrainment signal to all the agents in the network are analyzed and a model to account for them is proposed. △ Less

Submitted 2 September, 2015; originally announced September 2015.

Extended Hybrid Model Reference Adaptive Control of Piecewise Affine Systems

Authors: Mario di Bernardo, Umberto Montanaro, Romeo Ortega, Stefania Santini

Abstract: This note presents an extension to the adaptive control strategy presented in [1] able to counter eventual instability due to disturbances at the input of an otherwise $\mathcal{L}_2$ stable closed-loop system. These disturbances are due to the presence of affine terms in the plant and reference model. The existence of a common Lyapunov function is used to prove global convergence of the error sys… ▽ More This note presents an extension to the adaptive control strategy presented in [1] able to counter eventual instability due to disturbances at the input of an otherwise $\mathcal{L}_2$ stable closed-loop system. These disturbances are due to the presence of affine terms in the plant and reference model. The existence of a common Lyapunov function is used to prove global convergence of the error system, even in the presence of sliding solutions, as well as boundedness of all the adaptive gains. △ Less

Submitted 19 May, 2015; originally announced May 2015.

Comments: 14 pages

An algebraic metric for parametric stability analysis of power systems

Authors: Lewis Roberts, Alan Champneys, Keith Bell, Mario di Bernardo

Abstract: An analytic approximation for the critical clearing time (CCT) metric is derived from direct methods for power system stability. The formula has been designed to incorporate as many features of transient stability analysis as possible such as different fault locations and different post-fault network states. The purpose of this metric is to analyse trends in stability (in terms of CCT) of power sy… ▽ More An analytic approximation for the critical clearing time (CCT) metric is derived from direct methods for power system stability. The formula has been designed to incorporate as many features of transient stability analysis as possible such as different fault locations and different post-fault network states. The purpose of this metric is to analyse trends in stability (in terms of CCT) of power systems under the variation of a system parameter. We demonstrate the performance of this metric to measure stability trends on an aggregated power network, the so-called two machine infinite bus network, by varying load parameters in the full bus admittance matrix using numerical continuation. Our metric is compared to two other expressions for the CCT which incorporate additional non-linearities present in the model. △ Less

Submitted 19 May, 2015; v1 submitted 26 March, 2015; originally announced March 2015.

Comments: 11 pages

Contraction and incremental stability of switched Carathéodory systems using multiple norms

Authors: Wenlian Lu, Mario di Bernardo

Abstract: In this paper, incremental exponential asymptotic stability of a class of switched Carathéodory nonlinear systems is studied based on the novel concept of measure of switched matrices via multiple norms and the transaction coefficients between these norms. This model is rather general and includes the case of staircase switching signals as a special case. Sufficient conditions are derived for incr… ▽ More In this paper, incremental exponential asymptotic stability of a class of switched Carathéodory nonlinear systems is studied based on the novel concept of measure of switched matrices via multiple norms and the transaction coefficients between these norms. This model is rather general and includes the case of staircase switching signals as a special case. Sufficient conditions are derived for incremental stability allowing for the system to be incrementally exponentially asymptotically stable even if some of its modes are unstable in some time periods. Numerical examples on switched linear systems switching periodically and on the synchronization of switched networks of nonlinear systems are used to illustrate the theoretical results. △ Less

Submitted 29 February, 2016; v1 submitted 21 February, 2015; originally announced February 2015.

Comments: 10 pages, 4 figures

Distributed PID Control for Consensus of Homogeneous and Heterogeneous Networks

Authors: Daniel Burbano, Mario di Bernardo

Abstract: We investigate the use of distributed PID actions to achieve consensus in networks of homogeneous and heterogeneous linear systems. Convergence of the strategy is proved for both cases using appropriate state transformations and Lyapunov functions. The effectiveness of the theoretical results is illustrated via its application to a representative power grid model recently presented in the literatu… ▽ More We investigate the use of distributed PID actions to achieve consensus in networks of homogeneous and heterogeneous linear systems. Convergence of the strategy is proved for both cases using appropriate state transformations and Lyapunov functions. The effectiveness of the theoretical results is illustrated via its application to a representative power grid model recently presented in the literature. △ Less

Submitted 21 October, 2014; v1 submitted 8 September, 2014; originally announced September 2014.

Comments: 10 pages, 7 Figures, Accepted for Publication in IEEE Transactions on Control of Network Systems

MSC Class: 93A15 (Primary); 93A14 (Secondary) ACM Class: I.2.8; C.2.1

Convergence and synchronization in heterogeneous networks of smooth and piecewise smooth systems

Authors: Pietro DeLellis, Mario di Bernardo, Davide Liuzza

Abstract: This paper presents a framework for the study of convergence when the nodes' dynamics may be both piecewise smooth and/or nonidentical across the network. Specifically, we derive sufficient conditions for global convergence of all node trajectories towards the same bounded region of their state space. The analysis is based on the use of set-valued Lyapunov functions and bounds are derived on the m… ▽ More This paper presents a framework for the study of convergence when the nodes' dynamics may be both piecewise smooth and/or nonidentical across the network. Specifically, we derive sufficient conditions for global convergence of all node trajectories towards the same bounded region of their state space. The analysis is based on the use of set-valued Lyapunov functions and bounds are derived on the minimum coupling strength required to make all nodes in the network converge towards each other. We also provide an estimate of the asymptotic bound $ε$ on the mismatch between the node states at steady state. The analysis is performed both for linear and nonlinear coupling protocols. The theoretical analysis is extensively illustrated and validated via its application to a set of representative numerical examples. △ Less

Submitted 7 April, 2014; originally announced April 2014.

Comments: 50 pages, 11 figures

MSC Class: 93A14

Contraction analysis of switched systems: the case of Caratheodory Systems and Networks

Authors: Mario di Bernardo, Davide Liuzza, Giovanni Russo

Abstract: In this paper we extend to a generic class of piecewise smooth dynamical systems a fundamental tool for the analysis of convergence of smooth dynamical systems: contraction theory. We focus on switched systems satisfying Caratheodory conditions for the existence and unicity of a solution. After generalizing the classical definition of contraction to this class of dynamical systems, we give suffici… ▽ More In this paper we extend to a generic class of piecewise smooth dynamical systems a fundamental tool for the analysis of convergence of smooth dynamical systems: contraction theory. We focus on switched systems satisfying Caratheodory conditions for the existence and unicity of a solution. After generalizing the classical definition of contraction to this class of dynamical systems, we give sufficient conditions for global exponential convergence of their trajectories. The theoretical results are then applied to solve a set of representative problems including proving global asymptotic stability of switched linear systems, giving conditions for incremental stability of piecewise smooth systems, and analyzing the convergence of networked switched linear systems. △ Less

Submitted 4 October, 2011; originally announced October 2011.

Comments: 29 pages, 3 figures

Dynamics of symmetric dynamical systems with delayed switching

Authors: J. Sieber, P. Kowalczyk, S. J. Hogan, M. di Bernardo

Abstract: We study dynamical systems that switch between two different vector fields depending on a discrete variable and with a delay. When the delay reaches a problem-dependent critical value so-called event collisions occur. This paper classifies and analyzes event collisions, a special type of discontinuity induced bifurcations, for periodic orbits. Our focus is on event collisions of symmetric period… ▽ More We study dynamical systems that switch between two different vector fields depending on a discrete variable and with a delay. When the delay reaches a problem-dependent critical value so-called event collisions occur. This paper classifies and analyzes event collisions, a special type of discontinuity induced bifurcations, for periodic orbits. Our focus is on event collisions of symmetric periodic orbits in systems with full reflection symmetry, a symmetry that is prevalent in applications. We derive an implicit expression for the Poincare map near the colliding periodic orbit. The Poincare map is piecewise smooth, finite-dimensional, and changes the dimension of its image at the collision. In the second part of the paper we apply this general result to the class of unstable linear single-degree-of-freedom oscillators where we detect and continue numerically collisions of invariant tori. Moreover, we observe that attracting closed invariant polygons emerge at the torus collision. △ Less

Submitted 2 April, 2008; originally announced April 2008.

Comments: 28 pages

MSC Class: 37G15; 70K43; 70K50

Journal ref: Journal of Vibration and Control 16(7-8) pp. 1111-1140, 2010