-
Reinforcement Learning for Low-Thrust Trajectory Design of Interplanetary Missions
Authors:
Alessandro Zavoli,
Lorenzo Federici
Abstract:
This paper investigates the use of Reinforcement Learning for the robust design of low-thrust interplanetary trajectories in presence of severe disturbances, modeled alternatively as Gaussian additive process noise, observation noise, control actuation errors on thrust magnitude and direction, and possibly multiple missed thrust events. The optimal control problem is recast as a time-discrete Mark…
▽ More
This paper investigates the use of Reinforcement Learning for the robust design of low-thrust interplanetary trajectories in presence of severe disturbances, modeled alternatively as Gaussian additive process noise, observation noise, control actuation errors on thrust magnitude and direction, and possibly multiple missed thrust events. The optimal control problem is recast as a time-discrete Markov Decision Process to comply with the standard formulation of reinforcement learning. An open-source implementation of the state-of-the-art algorithm Proximal Policy Optimization is adopted to carry out the training process of a deep neural network, used to map the spacecraft (observed) states to the optimal control policy. The resulting Guidance and Control Network provides both a robust nominal trajectory and the associated closed-loop guidance law. Numerical results are presented for a typical Earth-Mars mission. First, in order to validate the proposed approach, the solution found in a (deterministic) unperturbed scenario is compared with the optimal one provided by an indirect technique. Then, the robustness and optimality of the obtained closed-loop guidance laws is assessed by means of Monte Carlo campaigns performed in the considered uncertain scenarios. These preliminary results open up new horizons for the use of reinforcement learning in the robust design of interplanetary missions.
△ Less
Submitted 19 August, 2020;
originally announced August 2020.
-
EOS: a Parallel, Self-Adaptive, Multi-Population Evolutionary Algorithm for Constrained Global Optimization
Authors:
Lorenzo Federici,
Boris Benedikter,
Alessandro Zavoli
Abstract:
This paper presents the main characteristics of the evolutionary optimization code named EOS, Evolutionary Optimization at Sapienza, and its successful application to challenging, real-world space trajectory optimization problems. EOS is a global optimization algorithm for constrained and unconstrained problems of real-valued variables. It implements a number of improvements to the well-known Diff…
▽ More
This paper presents the main characteristics of the evolutionary optimization code named EOS, Evolutionary Optimization at Sapienza, and its successful application to challenging, real-world space trajectory optimization problems. EOS is a global optimization algorithm for constrained and unconstrained problems of real-valued variables. It implements a number of improvements to the well-known Differential Evolution (DE) algorithm, namely, a self-adaptation of the control parameters, an epidemic mechanism, a clustering technique, an $\varepsilon$-constrained method to deal with nonlinear constraints, and a synchronous island-model to handle multiple populations in parallel. The results reported prove that EOSis capable of achieving increased performance compared to state-of-the-art single-population self-adaptive DE algorithms when applied to high-dimensional or highly-constrained space trajectory optimization problems.
△ Less
Submitted 13 July, 2020; v1 submitted 9 July, 2020;
originally announced July 2020.
-
Integrated Optimization of Ascent Trajectory and SRM Design of Multistage Launch Vehicles
Authors:
Lorenzo Federici,
Alessandro Zavoli,
Guido Colasurdo,
Lucandrea Mancini,
Agostino Neri
Abstract:
This paper presents a methodology for the concurrent first-stage preliminary design and ascent trajectory optimization, with application to a Vega-derived Light Launch Vehicle. The reuse as first stage of an existing upper-stage (Zefiro 40) requires a propellant grain geometry redesign, in order to account for the mutated operating conditions. An optimization code based on the parallel running of…
▽ More
This paper presents a methodology for the concurrent first-stage preliminary design and ascent trajectory optimization, with application to a Vega-derived Light Launch Vehicle. The reuse as first stage of an existing upper-stage (Zefiro 40) requires a propellant grain geometry redesign, in order to account for the mutated operating conditions. An optimization code based on the parallel running of several Differential Evolution algorithms is used to find the optimal internal pressure law during Z40 operation, together with the optimal thrust direction and other relevant flight parameters of the entire ascent trajectory. Payload injected into a target orbit is maximized, while respecting multiple design constraints, either involving the alone solid rocket motor or dependent on the actual flight trajectory. Numerical results for SSO injection are presented.
△ Less
Submitted 8 October, 2019;
originally announced October 2019.
-
A Time-Dependent TSP Formulation for the Design of an Active Debris Removal Mission using Simulated Annealing
Authors:
Lorenzo Federici,
Alessandro Zavoli,
Guido Colasurdo
Abstract:
This paper proposes a formulation of the Active Debris Removal (ADR) Mission Design problem as a modified Time-Dependent Traveling Salesman Problem (TDTSP). The TDTSP is a well-known combinatorial optimization problem, whose solution is the cheapest mono-cyclic tour connecting a number of non-stationary cities in a map. The problem is tackled with an optimization procedure based on Simulated Annea…
▽ More
This paper proposes a formulation of the Active Debris Removal (ADR) Mission Design problem as a modified Time-Dependent Traveling Salesman Problem (TDTSP). The TDTSP is a well-known combinatorial optimization problem, whose solution is the cheapest mono-cyclic tour connecting a number of non-stationary cities in a map. The problem is tackled with an optimization procedure based on Simulated Annealing, that efficiently exploits a natural encoding and a careful choice of mutation operators. The developed algorithm is used to simultaneously optimize the targets sequence and the rendezvous epochs of an impulsive ADR mission. Numerical results are presented for sets comprising up to 20 targets.
△ Less
Submitted 23 September, 2019;
originally announced September 2019.
-
A Checklist for the Evaluation of Pedestrian Simulation Software Functionalities
Authors:
Mizar Luca Federici,
Lorenza Manenti,
Sara Manzoni
Abstract:
The employment of micro-simulation (agent-based) tools in the phase of design of public and private spaces and facilities and for the definition of transport schemes that impact on pedestrian flows, thanks to their achieved accuracy and predictive capacity, has become a consolidated practice. These instruments provide support to the organization of spaces, services and facilities and to the defini…
▽ More
The employment of micro-simulation (agent-based) tools in the phase of design of public and private spaces and facilities and for the definition of transport schemes that impact on pedestrian flows, thanks to their achieved accuracy and predictive capacity, has become a consolidated practice. These instruments provide support to the organization of spaces, services and facilities and to the definition of management procedures for normal and emergency situations. The employment of these tools is effective for various but not for all the contexts, nevertheless new features and functions are under constant development and new products are often launched on the market. Therefore, there is a higher necessity of a standard criteria both for the evaluation of the kinds of function that these software provide, at use of practitioners and end-users, and for the definition of software requirements as a reference for the developers that aim at being competitive on this market.
On the basis of our experience as pedestrian modellers and as researchers in the crowd modelling area, we designed a comprehensive and detailed ready-to-use checklist for the quantitative evaluation of Pedestrian Simulation Software functionalities that aims at capturing all the aspects that we claim that are useful to undertake a professional study. These functions in our opinion are necessary to provide accurate results in the planning of new facilities or schemes that involve pedestrian activities. With this work we propose a set of criteria of evaluation for these products also to encourage a debate for the definition of objective standards for pedestrian simulation software certification.
△ Less
Submitted 17 July, 2014; v1 submitted 30 April, 2014;
originally announced April 2014.
