-
Adaptive higher order reversible integrators for memory efficient deep learning
Authors:
Sofya Maslovskaya,
Sina Ober-Blöbaum,
Christian Offen,
Pranav Singh,
Boris Wembe
Abstract:
The depth of networks plays a crucial role in the effectiveness of deep learning. However, the memory requirement for backpropagation scales linearly with the number of layers, which leads to memory bottlenecks during training. Moreover, deep networks are often unable to handle time-series data appearing at irregular intervals. These issues can be resolved by considering continuous-depth networks…
▽ More
The depth of networks plays a crucial role in the effectiveness of deep learning. However, the memory requirement for backpropagation scales linearly with the number of layers, which leads to memory bottlenecks during training. Moreover, deep networks are often unable to handle time-series data appearing at irregular intervals. These issues can be resolved by considering continuous-depth networks based on the neural ODE framework in combination with reversible integration methods that allow for variable time-steps. Reversibility of the method ensures that the memory requirement for training is independent of network depth, while variable time-steps are required for assimilating time-series data on irregular intervals. However, at present, there are no known higher-order reversible methods with this property. High-order methods are especially important when a high level of accuracy in learning is required or when small time-steps are necessary due to large errors in time integration of neural ODEs, for instance in context of complex dynamical systems such as Kepler systems and molecular dynamics. The requirement of small time-steps when using a low-order method can significantly increase the computational cost of training as well as inference. In this work, we present an approach for constructing high-order reversible methods that allow adaptive time-stepping. Our numerical tests show the advantages in computational speed when applied to the task of learning dynamical systems.
△ Less
Submitted 19 February, 2025; v1 submitted 12 October, 2024;
originally announced October 2024.
-
Commutator-free Cayley methods
Authors:
Sofya Maslovskaya,
Christian Offen,
Sina Ober-Blöbaum,
Pranav Singh,
Boris Wembe
Abstract:
Differential equations posed on quadratic matrix Lie groups arise in the context of classical mechanics and quantum dynamical systems. Lie group numerical integrators preserve the constants of motions defining the Lie group. Thus, they respect important physical laws of the dynamical system, such as unitarity and energy conservation in the context of quantum dynamical systems, for instance. In thi…
▽ More
Differential equations posed on quadratic matrix Lie groups arise in the context of classical mechanics and quantum dynamical systems. Lie group numerical integrators preserve the constants of motions defining the Lie group. Thus, they respect important physical laws of the dynamical system, such as unitarity and energy conservation in the context of quantum dynamical systems, for instance. In this article we develop a high-order commutator free Lie group integrator for non-autonomous differential equations evolving on quadratic Lie groups. Instead of matrix exponentials, which are expensive to evaluate and need to be approximated by appropriate rational functions in order to preserve the Lie group structure, the proposed method is obtained as a composition of Cayley transforms which naturally respect the structure of quadratic Lie groups while being computationally efficient to evaluate. Unlike Cayley-Magnus methods the method is also free from nested matrix commutators.
△ Less
Submitted 20 February, 2025; v1 submitted 23 August, 2024;
originally announced August 2024.
-
Trim turnpikes for optimal control problems with symmetries
Authors:
Kathrin Flaßkamp,
Sofya Maslovskaya,
Sina Ober-Blöbaum,
Boris Wembe
Abstract:
Motivated by mechanical systems with symmetries, we focus on optimal control problems possessing symmetries. Following recent works, which generalized the classical concept of static turnpike to manifold turnpike, we extend the exponential turnpike property to the exponential trim turnpike for control systems with symmetries induced by abelian or non-abelian groups. Our analysis is mainly based on…
▽ More
Motivated by mechanical systems with symmetries, we focus on optimal control problems possessing symmetries. Following recent works, which generalized the classical concept of static turnpike to manifold turnpike, we extend the exponential turnpike property to the exponential trim turnpike for control systems with symmetries induced by abelian or non-abelian groups. Our analysis is mainly based on the geometric reduction of control systems with symmetries. More concretely, we first reduce the control system on the quotient space and state the turnpike theorem for the reduced problem. Then we use the group properties to obtain the trim turnpike theorem for the full problem. Finally, we illustrate our results on the Kepler problem and the Rigid body problem.
△ Less
Submitted 21 June, 2024; v1 submitted 20 June, 2024;
originally announced June 2024.
-
Singular versus boundary arcs for aircraft trajectory optimization in climbing phase
Authors:
Olivier Cots,
Joseph Gergaud,
Damien Goubinat,
Boris Wembe
Abstract:
In this article, we are interested in optimal aircraft trajectories in climbing phase. We consider the cost index criterion which is a convex combination of the time-to-climb and the fuel consumption. We assume that the thrust is constant and we control the flight path angle of the aircraft. This optimization problem is modeled as a Mayer optimal control problem with a single-input affine dynamics…
▽ More
In this article, we are interested in optimal aircraft trajectories in climbing phase. We consider the cost index criterion which is a convex combination of the time-to-climb and the fuel consumption. We assume that the thrust is constant and we control the flight path angle of the aircraft. This optimization problem is modeled as a Mayer optimal control problem with a single-input affine dynamics in the control and with two pure state constraints, limiting the Calibrated AirSpeed (CAS) and the Mach speed. The candidates as minimizers are selected among a set of extremals given by the maximum principle. We first analyze the minimum time-to-climb problem with respect to the bounds of the state constraints, combining small time analysis, indirect multiple shooting and homotopy methods with monitoring. This investigation emphasizes two strategies: the common CAS/Mach procedure in aeronautics and the classical Bang-Singular-Bang policy in control theory. We then compare these two procedures for the cost index criterion.
△ Less
Submitted 13 December, 2022;
originally announced December 2022.
-
Accessibility Properties of Abnormal Geodesics in Optimal Control Illustrated by Two Case Studies
Authors:
Bernard Bonnard,
Jérémy Rouot,
Boris Wembe
Abstract:
In this article, we use two case studies from geometry and optimal control of chemical network to analyze the relation between abnormal geodesics in time optimal control, accessibility properties and regularity of the time minimal value function.
In this article, we use two case studies from geometry and optimal control of chemical network to analyze the relation between abnormal geodesics in time optimal control, accessibility properties and regularity of the time minimal value function.
△ Less
Submitted 23 February, 2022;
originally announced February 2022.
-
Abnormal Geodesics in 2D-Zermelo Navigation Problems in the Case of Revolution and the Fan Shape of the Small Time Balls
Authors:
Bernard Bonnard,
Olivier Cots,
Joseph Gergaud,
Boris Wembe
Abstract:
In this article, based on two case studies, we discuss the role of abnormal geodesics in planar Zermelo navigation problems. Such curves are limit curves of the accessibility set, in the domain where the current is strong. The problem is set in the frame of geometric time optimal control, where the control is the heading angle of the ship and in this context, abnormal curves are shown to separate…
▽ More
In this article, based on two case studies, we discuss the role of abnormal geodesics in planar Zermelo navigation problems. Such curves are limit curves of the accessibility set, in the domain where the current is strong. The problem is set in the frame of geometric time optimal control, where the control is the heading angle of the ship and in this context, abnormal curves are shown to separate time minimal curves from time maximal curves and are both small-time minimizing and maximizing. We describe the small-time minimal balls. For bigger time, a cusp singularity can occur in the abnormal direction, which corresponds to a conjugate point along the non-smooth image. It is interpreted in terms of the regularity property of the time minimal value function.
△ Less
Submitted 11 January, 2022;
originally announced January 2022.
-
A Zermelo navigation problem with a vortex singularity
Authors:
Bernard Bonnard,
Olivier Cots,
Boris Wembe
Abstract:
Helhmoltz-Kirchhoff equations of motions of vortices of an incompressible fluid in the plane define a dynamics with singularities and this leads to a Zermelo navigation problem describing the ship travel in such a field where the control is the heading angle. Considering one vortex, we define a time minimization problem which can be analyzed with the technics of geometric optimal control combined…
▽ More
Helhmoltz-Kirchhoff equations of motions of vortices of an incompressible fluid in the plane define a dynamics with singularities and this leads to a Zermelo navigation problem describing the ship travel in such a field where the control is the heading angle. Considering one vortex, we define a time minimization problem which can be analyzed with the technics of geometric optimal control combined with numerical simulations, the geometric frame being the extension of Randers metrics in the punctured plane, with rotational symmetry. Candidates as minimizers are parameterized thanks to the Pontryagin Maximum Principle as extremal solutions of a Hamiltonian vector field. We analyze the time minimal solution to transfer the ship between two points where during the transfer the ship can be either in a strong current region in the vicinity of the vortex or in a weak current region. The analysis is based on a micro-local classification of the extremals using mainly the integrability properties of the dynamics due to the rotational symmetry. The discussion is complex and related to the existence of an isolated extremal (Reeb) circle due to the vortex singularity. The explicit computation of cut points where the extremal curves cease to be optimal is given and the spheres are described in the case where at the initial point the current is weak.
△ Less
Submitted 13 July, 2020; v1 submitted 4 November, 2019;
originally announced November 2019.
