-
Data-Driven LQR with Finite-Time Experiments via Extremum-Seeking Policy Iteration
Authors:
Guido Carnevale,
Nicola Mimmo,
Giuseppe Notarstefano
Abstract:
In this paper, we address Linear Quadratic Regulator (LQR) problems through a novel iterative algorithm named EXtremum-seeking Policy iteration LQR (EXP-LQR). The peculiarity of EXP-LQR is that it only needs access to a truncated approximation of the infinite-horizon cost associated to a given policy. Hence, EXP-LQR does not need the direct knowledge of neither the system matrices, cost matrices,…
▽ More
In this paper, we address Linear Quadratic Regulator (LQR) problems through a novel iterative algorithm named EXtremum-seeking Policy iteration LQR (EXP-LQR). The peculiarity of EXP-LQR is that it only needs access to a truncated approximation of the infinite-horizon cost associated to a given policy. Hence, EXP-LQR does not need the direct knowledge of neither the system matrices, cost matrices, and state measurements. In particular, at each iteration, EXP-LQR refines the maintained policy using a truncated LQR cost retrieved by performing finite-time virtual or real experiments in which a perturbed version of the current policy is employed. Such a perturbation is done according to an extremum-seeking mechanism and makes the overall algorithm a time-varying nonlinear system. By using a Lyapunov-based approach exploiting averaging theory, we show that EXP-LQR exponentially converges to an arbitrarily small neighborhood of the optimal gain matrix. We corroborate the theoretical results with numerical simulations involving the control of an induction motor.
△ Less
Submitted 5 December, 2024; v1 submitted 3 December, 2024;
originally announced December 2024.
-
A Unifying System Theory Framework for Distributed Optimization and Games
Authors:
Guido Carnevale,
Nicola Mimmo,
Giuseppe Notarstefano
Abstract:
This paper introduces a systematic methodological framework to design and analyze distributed algorithms for optimization and games over networks. Starting from a centralized method, we identify an aggregation function involving all the decision variables (e.g., a global cost gradient or constraint) and introduce a distributed consensus-oriented scheme to asymptotically approximate the unavailable…
▽ More
This paper introduces a systematic methodological framework to design and analyze distributed algorithms for optimization and games over networks. Starting from a centralized method, we identify an aggregation function involving all the decision variables (e.g., a global cost gradient or constraint) and introduce a distributed consensus-oriented scheme to asymptotically approximate the unavailable information at each agent. Then, we delineate the proper methodology for intertwining the identified building blocks, i.e., the optimization-oriented method and the consensus-oriented one. The key intuition is to interpret the obtained interconnection as a singularly perturbed system. We rely on this interpretation to provide sufficient conditions for the building blocks to be successfully connected into a distributed scheme exhibiting the convergence guarantees of the centralized algorithm. Finally, we show the potential of our approach by developing a new distributed scheme for constraint-coupled problems with a linear convergence rate.
△ Less
Submitted 15 January, 2025; v1 submitted 23 January, 2024;
originally announced January 2024.
-
High-Performance Motorbike Lean Angle Estimation
Authors:
Nicola Mimmo,
Matteo Zanzi
Abstract:
This work deals with the real-time estimation of the lean angle of high-performance motorbikes. The estimate is obtained through measurements provided by an onboard inertial sensor and a GNSS receiver. A two-stage state observer, implementing a kinematic model developed under the novel assumption of coordinated manoeuvre, processes these measurements. A theoretical analysis demonstrates the observ…
▽ More
This work deals with the real-time estimation of the lean angle of high-performance motorbikes. The estimate is obtained through measurements provided by an onboard inertial sensor and a GNSS receiver. A two-stage state observer, implementing a kinematic model developed under the novel assumption of coordinated manoeuvre, processes these measurements. A theoretical analysis demonstrates the observer's stability, while a covariance analysis assesses the estimate's accuracy and error bounds. Finally, experimental results obtained on race-track tests and numerical comparisons, with competitive approaches, in simulated realistic scenarios show the superior performance of the proposed estimator.
△ Less
Submitted 13 February, 2023;
originally announced February 2023.
-
Nonconvex Distributed Feedback Optimization for Aggregative Cooperative Robotics
Authors:
Guido Carnevale,
Nicola Mimmo,
Giuseppe Notarstefano
Abstract:
Distributed aggregative optimization is a recently emerged framework in which the agents of a network want to minimize the sum of local objective functions, each one depending on the agent decision variable (e.g., the local position of a team of robots) and an aggregation of all the agents' variables (e.g., the team barycentre). In this paper, we address a distributed feedback optimization framewo…
▽ More
Distributed aggregative optimization is a recently emerged framework in which the agents of a network want to minimize the sum of local objective functions, each one depending on the agent decision variable (e.g., the local position of a team of robots) and an aggregation of all the agents' variables (e.g., the team barycentre). In this paper, we address a distributed feedback optimization framework in which agents implement a local (distributed) policy to reach a steady-state minimizing an aggregative cost function. We propose Aggregative Tracking Feedback, i.e., a novel distributed feedback optimization law in which each agent combines a closed-loop gradient flow with a consensus-based dynamic compensator reconstructing the missing global information. By using tools from system theory, we prove that Aggregative Tracking Feedback steers the network to a stationary point of an aggregative optimization problem with (possibly) nonconvex objective function. The effectiveness of the proposed method is validated through numerical simulations on a multi-robot surveillance scenario.
△ Less
Submitted 5 April, 2024; v1 submitted 3 February, 2023;
originally announced February 2023.
-
Very large-scale neighborhood search for drone routing with energy replenishment
Authors:
Catherine Lorenz,
Nicola Mimmo,
Alena Otto,
Daniele Vigo
Abstract:
The Drone Routing Problem with Energy replenishment (DRP-E) belongs to a general class of routing problems with intermediate stops and synchronization constraints. In DRP-E, the drone has to visit a set of nodes and routinely requires battery swaps from a (potentially) mobile replenishment station. Contrary to widespread restrictions in the drone routing literature, several destinations may be vis…
▽ More
The Drone Routing Problem with Energy replenishment (DRP-E) belongs to a general class of routing problems with intermediate stops and synchronization constraints. In DRP-E, the drone has to visit a set of nodes and routinely requires battery swaps from a (potentially) mobile replenishment station. Contrary to widespread restrictions in the drone routing literature, several destinations may be visited in between two consecutive battery swaps. In this paper, we propose a nontrivial very large-scale neighbourhood for DRP-E, which synergetically leverages two large-sized polynomially solvable DRP-E SubProblems (SP1 and SP2). The number of feasible solutions in the resulting neighborhood is a multiple of those in SP1 and SP2, and, thus, exponential in the input size of the problem, whereas the computational time to search it remains polynomial. The proposed polynomial two-stage dynamic programming algorithm VLSN to search this neighborhood can be flexibly adjusted to the desired trade-off between accuracy and computational time. For instance, the search procedure can be converted into an exact algorithm of competitive runtime for DRP-E. In computational tests, the developed solution methods outperform current state-of-the art heuristics for DRP-E by a significant margin. A case study based on a search for missing persons demonstrates that VLSN easily accommodates additional practice relevant features and outperforms the state-of-the-art solution in disaster relief by 20%.
△ Less
Submitted 28 July, 2022;
originally announced July 2022.
-
Uniform non-convex optimisation via Extremum Seeking
Authors:
Nicola Mimmo,
Lorenzo Marconi,
Giuseppe Notarstefano
Abstract:
The paper deals with a well-known extremum seeking scheme by proving uniformity properties with respect to the amplitudes of the dither signal and of the cost function. Those properties are then used to show that the scheme guarantees the global minimiser to be semi-global practically stable despite the presence of local minima. Under the assumption of a globally Lipschitz cost function, it is sho…
▽ More
The paper deals with a well-known extremum seeking scheme by proving uniformity properties with respect to the amplitudes of the dither signal and of the cost function. Those properties are then used to show that the scheme guarantees the global minimiser to be semi-global practically stable despite the presence of local minima. Under the assumption of a globally Lipschitz cost function, it is shown that the scheme, improved through a high-pass filter, makes the global minimiser practically stable with a global domain of attraction.
△ Less
Submitted 5 June, 2022;
originally announced June 2022.
-
Uniform quasi-convex optimisation via Extremum Seeking
Authors:
Nicola Mimmo,
Lorenzo Marconi,
Giuseppe Notarstefano
Abstract:
The paper deals with a well-known extremum seeking scheme by proving uniformity properties with respect to the amplitudes of the dither signal and of the cost function. Those properties are then used to show that the scheme guarantees the global minimiser to be semi-global practically stable despite the presence of local saddle points. To achieve these results, we analyse the average system associ…
▽ More
The paper deals with a well-known extremum seeking scheme by proving uniformity properties with respect to the amplitudes of the dither signal and of the cost function. Those properties are then used to show that the scheme guarantees the global minimiser to be semi-global practically stable despite the presence of local saddle points. To achieve these results, we analyse the average system associated with the extremum seeking scheme via arguments based on the Fourier series.
△ Less
Submitted 1 April, 2022;
originally announced April 2022.
-
Extremum Seeking Tracking for Derivative-free Distributed Optimization
Authors:
Nicola Mimmo,
Guido Carnevale,
Andrea Testa,
Giuseppe Notarstefano
Abstract:
In this paper, we deal with a network of agents that want to cooperatively minimize the sum of local cost functions depending on a common decision variable. We consider the challenging scenario in which objective functions are unknown and agents have only access to local measurements of their local functions. We propose a novel distributed algorithm that combines a recent gradient tracking policy…
▽ More
In this paper, we deal with a network of agents that want to cooperatively minimize the sum of local cost functions depending on a common decision variable. We consider the challenging scenario in which objective functions are unknown and agents have only access to local measurements of their local functions. We propose a novel distributed algorithm that combines a recent gradient tracking policy with an extremum seeking technique to estimate the global descent direction. The joint use of these two techniques results in a distributed optimization scheme that provides arbitrarily accurate solution estimates through the combination of Lyapunov and averaging analysis approaches with consensus theory. We perform numerical simulations in a personalized optimization framework to corroborate the theoretical results.
△ Less
Submitted 7 November, 2024; v1 submitted 8 October, 2021;
originally announced October 2021.
