Showing 1–13 of 13 results for author: Eidnes, S

Learning Dynamical Systems from Noisy Data with Inverse-Explicit Integrators

Authors: Håkon Noren, Sølve Eidnes, Elena Celledoni

Abstract: We introduce the mean inverse integrator (MII), a novel approach to increase the accuracy when training neural networks to approximate vector fields of dynamical systems from noisy data. This method can be used to average multiple trajectories obtained by numerical integrators such as Runge-Kutta methods. We show that the class of mono-implicit Runge-Kutta methods (MIRK) has particular advantages… ▽ More We introduce the mean inverse integrator (MII), a novel approach to increase the accuracy when training neural networks to approximate vector fields of dynamical systems from noisy data. This method can be used to average multiple trajectories obtained by numerical integrators such as Runge-Kutta methods. We show that the class of mono-implicit Runge-Kutta methods (MIRK) has particular advantages when used in connection with MII. When training vector field approximations, explicit expressions for the loss functions are obtained when inserting the training data in the MIRK formulae, unlocking symmetric and high-order integrators that would otherwise be implicit for initial value problems. The combined approach of applying MIRK within MII yields a significantly lower error compared to the plain use of the numerical integrator without averaging the trajectories. This is demonstrated with experiments using data from several (chaotic) Hamiltonian systems. Additionally, we perform a sensitivity analysis of the loss functions under normally distributed perturbations, supporting the favorable performance of MII. △ Less

Submitted 6 June, 2023; originally announced June 2023.

Comments: 23 pages, 10 figures

Pseudo-Hamiltonian system identification

Authors: Sigurd Holmsen, Sølve Eidnes, Signe Riemer-Sørensen

Abstract: Identifying the underlying dynamics of physical systems can be challenging when only provided with observational data. In this work, we consider systems that can be modelled as first-order ordinary differential equations. By assuming a certain pseudo-Hamiltonian formulation, we are able to learn the analytic terms of internal dynamics even if the model is trained on data where the system is affect… ▽ More Identifying the underlying dynamics of physical systems can be challenging when only provided with observational data. In this work, we consider systems that can be modelled as first-order ordinary differential equations. By assuming a certain pseudo-Hamiltonian formulation, we are able to learn the analytic terms of internal dynamics even if the model is trained on data where the system is affected by unknown damping and external disturbances. In cases where it is difficult to find analytic terms for the disturbances, a hybrid model that uses a neural network to learn these can still accurately identify the dynamics of the system as if under ideal conditions. This makes the models applicable in some situations where other system identification models fail. Furthermore, we propose to use a fourth-order symmetric integration scheme in the loss function and avoid actual integration in the training, and demonstrate on varied examples how this leads to increased performance on noisy data. △ Less

Submitted 2 January, 2024; v1 submitted 9 May, 2023; originally announced May 2023.

Comments: 33 pages, 21 figures, including the appendix; v2: restructured and modified the text, added Section 6

MSC Class: Primary: 34A55; 37M10; Secondary: 37J99

Pseudo-Hamiltonian neural networks for learning partial differential equations

Authors: Sølve Eidnes, Kjetil Olsen Lye

Abstract: Pseudo-Hamiltonian neural networks (PHNN) were recently introduced for learning dynamical systems that can be modelled by ordinary differential equations. In this paper, we extend the method to partial differential equations. The resulting model is comprised of up to three neural networks, modelling terms representing conservation, dissipation and external forces, and discrete convolution operator… ▽ More Pseudo-Hamiltonian neural networks (PHNN) were recently introduced for learning dynamical systems that can be modelled by ordinary differential equations. In this paper, we extend the method to partial differential equations. The resulting model is comprised of up to three neural networks, modelling terms representing conservation, dissipation and external forces, and discrete convolution operators that can either be learned or be given as input. We demonstrate numerically the superior performance of PHNN compared to a baseline model that models the full dynamics by a single neural network. Moreover, since the PHNN model consists of three parts with different physical interpretations, these can be studied separately to gain insight into the system, and the learned model is applicable also if external forces are removed or changed. △ Less

Submitted 2 January, 2024; v1 submitted 27 April, 2023; originally announced April 2023.

Comments: 39 pages, 18 figures; v3: expanded text and added numerical experiments, new subsections: 5.1, 6.3, 6.4

Pseudo-Hamiltonian Neural Networks with State-Dependent External Forces

Authors: Sølve Eidnes, Alexander J. Stasik, Camilla Sterud, Eivind Bøhn, Signe Riemer-Sørensen

Abstract: Hybrid machine learning based on Hamiltonian formulations has recently been successfully demonstrated for simple mechanical systems, both energy conserving and not energy conserving. We introduce a pseudo-Hamiltonian formulation that is a generalization of the Hamiltonian formulation via the port-Hamiltonian formulation, and show that pseudo-Hamiltonian neural network models can be used to learn e… ▽ More Hybrid machine learning based on Hamiltonian formulations has recently been successfully demonstrated for simple mechanical systems, both energy conserving and not energy conserving. We introduce a pseudo-Hamiltonian formulation that is a generalization of the Hamiltonian formulation via the port-Hamiltonian formulation, and show that pseudo-Hamiltonian neural network models can be used to learn external forces acting on a system. We argue that this property is particularly useful when the external forces are state dependent, in which case it is the pseudo-Hamiltonian structure that facilitates the separation of internal and external forces. Numerical results are provided for a forced and damped mass-spring system and a tank system of higher complexity, and a symmetric fourth-order integration scheme is introduced for improved training on sparse and noisy data. △ Less

Submitted 23 January, 2023; v1 submitted 6 June, 2022; originally announced June 2022.

Comments: 23 pages, 13 figures; v4: slight title change, expanded on methodology for more clarity, updated plots

Order theory for discrete gradient methods

Authors: Sølve Eidnes

Abstract: The discrete gradient methods are integrators designed to preserve invariants of ordinary differential equations. From a formal series expansion of a subclass of these methods, we derive conditions for arbitrarily high order. We derive specific results for the average vector field discrete gradient, from which we get P-series methods in the general case, and B-series methods for canonical Hamilton… ▽ More The discrete gradient methods are integrators designed to preserve invariants of ordinary differential equations. From a formal series expansion of a subclass of these methods, we derive conditions for arbitrarily high order. We derive specific results for the average vector field discrete gradient, from which we get P-series methods in the general case, and B-series methods for canonical Hamiltonian systems. Higher order schemes are presented, and their applications are demonstrated on the Hénon-Heiles system and a Lotka-Volterra system, and on both the training and integration of a pendulum system learned from data by a neural network. △ Less

Submitted 15 January, 2022; v1 submitted 18 March, 2020; originally announced March 2020.

Comments: 49 pages, 7 figures; v3: Corrected typos and minor mistakes, added application to Hamiltonian neural networks; v4: Minor corrections

MSC Class: 65L05; 65P10; 37M15; 05C05

Linearly implicit local and global energy-preserving methods for PDEs with a cubic Hamiltonian

Authors: Sølve Eidnes, Lu Li

Abstract: We present linearly implicit methods that preserve discrete approximations to local and global energy conservation laws for multi-symplectic PDEs with cubic invariants. The methods are tested on the one-dimensional Korteweg-de Vries equation and the two-dimensional Zakharov-Kuznetsov equation; the numerical simulations confirm the conservative properties of the methods, and demonstrate their good… ▽ More We present linearly implicit methods that preserve discrete approximations to local and global energy conservation laws for multi-symplectic PDEs with cubic invariants. The methods are tested on the one-dimensional Korteweg-de Vries equation and the two-dimensional Zakharov-Kuznetsov equation; the numerical simulations confirm the conservative properties of the methods, and demonstrate their good stability properties and superior running speed when compared to fully implicit schemes. △ Less

Submitted 12 July, 2020; v1 submitted 3 July, 2019; originally announced July 2019.

Comments: 28 pages, 8 figures, 16 subfigures

MSC Class: 37K05; 65M06; 65P10

Linearly implicit structure-preserving schemes for Hamiltonian systems

Authors: Sølve Eidnes, Lu Li, Shun Sato

Abstract: Kahan's method and a two-step generalization of the discrete gradient method are both linearly implicit methods that can preserve a modified energy for Hamiltonian systems with a cubic Hamiltonian. These methods are here investigated and compared. The schemes are applied to the Korteweg-de Vries equation and the Camassa-Holm equation, and the numerical results are presented and analysed. Kahan's method and a two-step generalization of the discrete gradient method are both linearly implicit methods that can preserve a modified energy for Hamiltonian systems with a cubic Hamiltonian. These methods are here investigated and compared. The schemes are applied to the Korteweg-de Vries equation and the Camassa-Holm equation, and the numerical results are presented and analysed. △ Less

Submitted 8 May, 2020; v1 submitted 11 January, 2019; originally announced January 2019.

Comments: 18 pages, 11 figures, 33 subfigures. Submitted to Journal of Computational and Applied Mathematics, proceedings for NUMDIFF-15

MSC Class: 37K05; 82-08

Energy preserving methods on Riemannian manifolds

Authors: Elena Celledoni, Sølve Eidnes, Brynjulf Owren, Torbjørn Ringholm

Abstract: The energy preserving discrete gradient methods are generalized to finite-dimensional Riemannian manifolds by definition of a discrete approximation to the Riemannian gradient, a retraction, and a coordinate center function. The resulting schemes are intrinsic and do not depend on a particular choice of coordinates, nor on embedding of the manifold in a Euclidean space. Generalizations of well-kno… ▽ More The energy preserving discrete gradient methods are generalized to finite-dimensional Riemannian manifolds by definition of a discrete approximation to the Riemannian gradient, a retraction, and a coordinate center function. The resulting schemes are intrinsic and do not depend on a particular choice of coordinates, nor on embedding of the manifold in a Euclidean space. Generalizations of well-known discrete gradient methods, such as the average vector field method and the Itoh--Abe method are obtained. It is shown how methods of higher order can be constructed via a collocation-like approach. Local and global error bounds are derived in terms of the Riemannian distance function and the Levi-Civita connection. Some numerical results on spin system problems are presented. △ Less

Submitted 19 May, 2018; originally announced May 2018.

Comments: 17 pages, 3 figures, 6 subfigures

MSC Class: 37K05; 53B99; 65L05; 82-08

Dissipative numerical schemes on Riemannian manifolds with applications to gradient flows

Authors: Elena Celledoni, Sølve Eidnes, Brynjulf Owren, Torbjørn Ringholm

Abstract: This paper concerns an extension of discrete gradient methods to finite-dimensional Riemannian manifolds termed discrete Riemannian gradients, and their application to dissipative ordinary differential equations. This includes Riemannian gradient flow systems which occur naturally in optimization problems. The Itoh--Abe discrete gradient is formulated and applied to gradient systems, yielding a de… ▽ More This paper concerns an extension of discrete gradient methods to finite-dimensional Riemannian manifolds termed discrete Riemannian gradients, and their application to dissipative ordinary differential equations. This includes Riemannian gradient flow systems which occur naturally in optimization problems. The Itoh--Abe discrete gradient is formulated and applied to gradient systems, yielding a derivative-free optimization algorithm. The algorithm is tested on two eigenvalue problems and two problems from manifold valued imaging: InSAR denoising and DTI denoising. △ Less

Submitted 9 October, 2018; v1 submitted 22 April, 2018; originally announced April 2018.

Comments: Post-revision version. To appear in SIAM Journal on Scientific Computing

MSC Class: 49M37; 53B99; 65K10; 92C55; 90C26; 90C30; 90C56

Energy preserving moving mesh methods applied to the BBM equation

Authors: Sølve Eidnes, Torbjørn Ringholm

Abstract: Energy preserving numerical methods for a certain class of PDEs are derived, applying the partition of unity method. The methods are extended to also be applicable in combination with moving mesh methods by the rezoning approach. These energy preserving moving mesh methods are then applied to the Benjamin--Bona--Mahony equation, resulting in schemes that exactly preserve an approximation to one of… ▽ More Energy preserving numerical methods for a certain class of PDEs are derived, applying the partition of unity method. The methods are extended to also be applicable in combination with moving mesh methods by the rezoning approach. These energy preserving moving mesh methods are then applied to the Benjamin--Bona--Mahony equation, resulting in schemes that exactly preserve an approximation to one of the Hamiltonians of the system. Numerical experiments that demonstrate the advantages of the methods are presented. △ Less

Submitted 3 October, 2017; originally announced October 2017.

Comments: 13 pages; 7 figures, 13 subfigures

MSC Class: 65M50; 65M06; 65M60

Journal ref: MekIT '17 - Ninth national conference on Computational Mechanics, International Center for Numerical Methods in Engineering (CIMNE), 2017, ISBN 978-84-947311-1-2, p. 121-136

Shape analysis on Lie groups and homogeneous spaces

Authors: Elena Celledoni, Sølve Eidnes, Markus Eslitzbichler, Alexander Schmeding

Abstract: In this paper we are concerned with the approach to shape analysis based on the so called Square Root Velocity Transform (SRVT). We propose a generalisation of the SRVT from Euclidean spaces to shape spaces of curves on Lie groups and on homogeneous manifolds. The main idea behind our approach is to exploit the geometry of the natural Lie group actions on these spaces. In this paper we are concerned with the approach to shape analysis based on the so called Square Root Velocity Transform (SRVT). We propose a generalisation of the SRVT from Euclidean spaces to shape spaces of curves on Lie groups and on homogeneous manifolds. The main idea behind our approach is to exploit the geometry of the natural Lie group actions on these spaces. △ Less

Submitted 2 October, 2017; originally announced October 2017.

Comments: 8 pages, Contribution to the conference "Geometric Science of Information '17"

MSC Class: 58D15 (Primary); 58D10; 22E65; 58B20 (Secondary)

Journal ref: Geometric science of information, Lecture Notes in Comput. Sci., 10589, 2017

Shape analysis on homogeneous spaces: a generalised SRVT framework

Authors: Elena Celledoni, Sølve Eidnes, Alexander Schmeding

Abstract: Shape analysis is ubiquitous in problems of pattern and object recognition and has developed considerably in the last decade. The use of shapes is natural in applications where one wants to compare curves independently of their parametrisation. One computationally efficient approach to shape analysis is based on the Square Root Velocity Transform (SRVT). In this paper we propose a generalised SRVT… ▽ More Shape analysis is ubiquitous in problems of pattern and object recognition and has developed considerably in the last decade. The use of shapes is natural in applications where one wants to compare curves independently of their parametrisation. One computationally efficient approach to shape analysis is based on the Square Root Velocity Transform (SRVT). In this paper we propose a generalised SRVT framework for shapes on homogeneous manifolds. The method opens up for a variety of possibilities based on different choices of Lie group action and giving rise to different Riemannian metrics. △ Less

Submitted 26 March, 2018; v1 submitted 5 April, 2017; originally announced April 2017.

Comments: 28 pages; 4 figures, 30 subfigures; notes for proceedings of the Abel Symposium 2016: "Computation and Combinatorics in Dynamics, Stochastics and Control". v3: amended the text to improve readability and clarify some points; updated and added some references; added pseudocode for the dynamic programming algorithm used. The main results remain unchanged

MSC Class: 58D15 (Primary); 58D10; 22E65; 58B20 (Secondary)

Journal ref: E. Celledoni et al. (eds.), Computation and Combinatorics in Dynamics, Stochastics and Control, Abelsymposium 2016, Abel Symposia 13

Adaptive Energy Preserving Methods for Partial Differential Equations

Authors: Sølve Eidnes, Brynjulf Owren, Torbjørn Ringholm

Abstract: A method for constructing first integral preserving numerical schemes for time-dependent partial differential equations on non-uniform grids is presented. The method can be used with both finite difference and partition of unity approaches, thereby also including finite element approaches. The schemes are then extended to accommodate $r$-, $h$- and $p$-adaptivity. The method is applied to the Kort… ▽ More A method for constructing first integral preserving numerical schemes for time-dependent partial differential equations on non-uniform grids is presented. The method can be used with both finite difference and partition of unity approaches, thereby also including finite element approaches. The schemes are then extended to accommodate $r$-, $h$- and $p$-adaptivity. The method is applied to the Korteweg-de Vries equation and the Sine-Gordon equation and results from numerical experiments are presented. △ Less

Submitted 1 June, 2018; v1 submitted 9 July, 2015; originally announced July 2015.

Comments: 27 pages; some changes to notation and figures