-
Recency-Weighted Temporally-Segmented Ensemble for Time-Series Modeling
Authors:
Pål V. Johnsen,
Eivind Bøhn,
Sølve Eidnes,
Filippo Remonato,
Signe Riemer-Sørensen
Abstract:
Time-series modeling in process industries faces the challenge of dealing with complex, multi-faceted, and evolving data characteristics. Conventional single model approaches often struggle to capture the interplay of diverse dynamics, resulting in suboptimal forecasts. Addressing this, we introduce the Recency-Weighted Temporally-Segmented (ReWTS, pronounced `roots') ensemble model, a novel chunk…
▽ More
Time-series modeling in process industries faces the challenge of dealing with complex, multi-faceted, and evolving data characteristics. Conventional single model approaches often struggle to capture the interplay of diverse dynamics, resulting in suboptimal forecasts. Addressing this, we introduce the Recency-Weighted Temporally-Segmented (ReWTS, pronounced `roots') ensemble model, a novel chunk-based approach for multi-step forecasting. The key characteristics of the ReWTS model are twofold: 1) It facilitates specialization of models into different dynamics by segmenting the training data into `chunks' of data and training one model per chunk. 2) During inference, an optimization procedure assesses each model on the recent past and selects the active models, such that the appropriate mixture of previously learned dynamics can be recalled to forecast the future. This method not only captures the nuances of each period, but also adapts more effectively to changes over time compared to conventional `global' models trained on all data in one go. We present a comparative analysis, utilizing two years of data from a wastewater treatment plant and a drinking water treatment plant in Norway, demonstrating the ReWTS ensemble's superiority. It consistently outperforms the global model in terms of mean squared forecasting error across various model architectures by 10-70\% on both datasets, notably exhibiting greater resilience to outliers. This approach shows promise in developing automatic, adaptable forecasting models for decision-making and control systems in process industries and other complex systems.
△ Less
Submitted 4 March, 2024;
originally announced March 2024.
-
Road Graph Generator: Mapping roads at construction sites from GPS data
Authors:
Katarzyna Michałowska,
Helga Margrete Bodahl Holmestad,
Signe Riemer-Sørensen
Abstract:
We propose a new method for inferring roads from GPS trajectories to map construction sites. This task presents a unique challenge due to the erratic and non-standard movement patterns of construction machinery, which significantly diverge from typical vehicular traffic on established roads. Our proposed method first identifies intersections in the road network that serve as critical decision poin…
▽ More
We propose a new method for inferring roads from GPS trajectories to map construction sites. This task presents a unique challenge due to the erratic and non-standard movement patterns of construction machinery, which significantly diverge from typical vehicular traffic on established roads. Our proposed method first identifies intersections in the road network that serve as critical decision points, and then connects them with edges to produce a graph, which can subsequently be used for planning and task-allocation. We demonstrate the approach by mapping roads at a real-life construction site in Norway. The method is validated on four increasingly complex segments of the map. In our tests, the method achieved perfect accuracy in detecting intersections and inferring roads in data with no or low noise, while its performance was reduced in areas with significant noise and consistently missing GPS updates.
△ Less
Submitted 8 October, 2024; v1 submitted 15 February, 2024;
originally announced February 2024.
-
DON-LSTM: Multi-Resolution Learning with DeepONets and Long Short-Term Memory Neural Networks
Authors:
Katarzyna Michałowska,
Somdatta Goswami,
George Em Karniadakis,
Signe Riemer-Sørensen
Abstract:
Deep operator networks (DeepONets, DONs) offer a distinct advantage over traditional neural networks in their ability to be trained on multi-resolution data. This property becomes especially relevant in real-world scenarios where high-resolution measurements are difficult to obtain, while low-resolution data is more readily available. Nevertheless, DeepONets alone often struggle to capture and mai…
▽ More
Deep operator networks (DeepONets, DONs) offer a distinct advantage over traditional neural networks in their ability to be trained on multi-resolution data. This property becomes especially relevant in real-world scenarios where high-resolution measurements are difficult to obtain, while low-resolution data is more readily available. Nevertheless, DeepONets alone often struggle to capture and maintain dependencies over long sequences compared to other state-of-the-art algorithms. We propose a novel architecture, named DON-LSTM, which extends the DeepONet with a long short-term memory network (LSTM). Combining these two architectures, we equip the network with explicit mechanisms to leverage multi-resolution data, as well as capture temporal dependencies in long sequences. We test our method on long-time-evolution modeling of multiple non-linear systems and show that the proposed multi-resolution DON-LSTM achieves significantly lower generalization error and requires fewer high-resolution samples compared to its vanilla counterparts.
△ Less
Submitted 3 October, 2023;
originally announced October 2023.
-
Pseudo-Hamiltonian system identification
Authors:
Sigurd Holmsen,
Sølve Eidnes,
Signe Riemer-Sørensen
Abstract:
Identifying the underlying dynamics of physical systems can be challenging when only provided with observational data. In this work, we consider systems that can be modelled as first-order ordinary differential equations. By assuming a certain pseudo-Hamiltonian formulation, we are able to learn the analytic terms of internal dynamics even if the model is trained on data where the system is affect…
▽ More
Identifying the underlying dynamics of physical systems can be challenging when only provided with observational data. In this work, we consider systems that can be modelled as first-order ordinary differential equations. By assuming a certain pseudo-Hamiltonian formulation, we are able to learn the analytic terms of internal dynamics even if the model is trained on data where the system is affected by unknown damping and external disturbances. In cases where it is difficult to find analytic terms for the disturbances, a hybrid model that uses a neural network to learn these can still accurately identify the dynamics of the system as if under ideal conditions. This makes the models applicable in some situations where other system identification models fail. Furthermore, we propose to use a fourth-order symmetric integration scheme in the loss function and avoid actual integration in the training, and demonstrate on varied examples how this leads to increased performance on noisy data.
△ Less
Submitted 2 January, 2024; v1 submitted 9 May, 2023;
originally announced May 2023.
-
Neural Operator Learning for Long-Time Integration in Dynamical Systems with Recurrent Neural Networks
Authors:
Katarzyna Michałowska,
Somdatta Goswami,
George Em Karniadakis,
Signe Riemer-Sørensen
Abstract:
Deep neural networks are an attractive alternative for simulating complex dynamical systems, as in comparison to traditional scientific computing methods, they offer reduced computational costs during inference and can be trained directly from observational data. Existing methods, however, cannot extrapolate accurately and are prone to error accumulation in long-time integration. Herein, we addres…
▽ More
Deep neural networks are an attractive alternative for simulating complex dynamical systems, as in comparison to traditional scientific computing methods, they offer reduced computational costs during inference and can be trained directly from observational data. Existing methods, however, cannot extrapolate accurately and are prone to error accumulation in long-time integration. Herein, we address this issue by combining neural operators with recurrent neural networks, learning the operator mapping, while offering a recurrent structure to capture temporal dependencies. The integrated framework is shown to stabilize the solution and reduce error accumulation for both interpolation and extrapolation of the Korteweg-de Vries equation.
△ Less
Submitted 30 April, 2024; v1 submitted 3 March, 2023;
originally announced March 2023.
-
Pseudo-Hamiltonian Neural Networks with State-Dependent External Forces
Authors:
Sølve Eidnes,
Alexander J. Stasik,
Camilla Sterud,
Eivind Bøhn,
Signe Riemer-Sørensen
Abstract:
Hybrid machine learning based on Hamiltonian formulations has recently been successfully demonstrated for simple mechanical systems, both energy conserving and not energy conserving. We introduce a pseudo-Hamiltonian formulation that is a generalization of the Hamiltonian formulation via the port-Hamiltonian formulation, and show that pseudo-Hamiltonian neural network models can be used to learn e…
▽ More
Hybrid machine learning based on Hamiltonian formulations has recently been successfully demonstrated for simple mechanical systems, both energy conserving and not energy conserving. We introduce a pseudo-Hamiltonian formulation that is a generalization of the Hamiltonian formulation via the port-Hamiltonian formulation, and show that pseudo-Hamiltonian neural network models can be used to learn external forces acting on a system. We argue that this property is particularly useful when the external forces are state dependent, in which case it is the pseudo-Hamiltonian structure that facilitates the separation of internal and external forces. Numerical results are provided for a forced and damped mass-spring system and a tank system of higher complexity, and a symmetric fourth-order integration scheme is introduced for improved training on sparse and noisy data.
△ Less
Submitted 23 January, 2023; v1 submitted 6 June, 2022;
originally announced June 2022.
-
Mutual information estimation for graph convolutional neural networks
Authors:
Marius C. Landverk,
Signe Riemer-Sørensen
Abstract:
Measuring model performance is a key issue for deep learning practitioners. However, we often lack the ability to explain why a specific architecture attains superior predictive accuracy for a given data set. Often, validation accuracy is used as a performance heuristic quantifying how well a network generalizes to unseen data, but it does not capture anything about the information flow in the mod…
▽ More
Measuring model performance is a key issue for deep learning practitioners. However, we often lack the ability to explain why a specific architecture attains superior predictive accuracy for a given data set. Often, validation accuracy is used as a performance heuristic quantifying how well a network generalizes to unseen data, but it does not capture anything about the information flow in the model. Mutual information can be used as a measure of the quality of internal representations in deep learning models, and the information plane may provide insights into whether the model exploits the available information in the data. The information plane has previously been explored for fully connected neural networks and convolutional architectures. We present an architecture-agnostic method for tracking a network's internal representations during training, which are then used to create the mutual information plane. The method is exemplified for graph-based neural networks fitted on citation data. We compare how the inductive bias introduced in graph-based architectures changes the mutual information plane relative to a fully connected neural network.
△ Less
Submitted 31 March, 2022;
originally announced March 2022.
-
Inferring feature importance with uncertainties in high-dimensional data
Authors:
Pål Vegard Johnsen,
Inga Strümke,
Signe Riemer-Sørensen,
Andrew Thomas DeWan,
Mette Langaas
Abstract:
Estimating feature importance is a significant aspect of explaining data-based models. Besides explaining the model itself, an equally relevant question is which features are important in the underlying data generating process. We present a Shapley value based framework for inferring the importance of individual features, including uncertainty in the estimator. We build upon the recently published…
▽ More
Estimating feature importance is a significant aspect of explaining data-based models. Besides explaining the model itself, an equally relevant question is which features are important in the underlying data generating process. We present a Shapley value based framework for inferring the importance of individual features, including uncertainty in the estimator. We build upon the recently published feature importance measure of SAGE (Shapley additive global importance) and introduce sub-SAGE which can be estimated without resampling for tree-based models. We argue that the uncertainties can be estimated from bootstrapping and demonstrate the approach for tree ensemble methods. The framework is exemplified on synthetic data as well as high-dimensional genomics data.
△ Less
Submitted 20 September, 2021; v1 submitted 2 September, 2021;
originally announced September 2021.
-
Deep Reinforcement Learning for Long Term Hydropower Production Scheduling
Authors:
Signe Riemer-Sorensen,
Gjert H. Rosenlund
Abstract:
We explore the use of deep reinforcement learning to provide strategies for long term scheduling of hydropower production. We consider a use-case where the aim is to optimise the yearly revenue given week-by-week inflows to the reservoir and electricity prices. The challenge is to decide between immediate water release at the spot price of electricity and storing the water for later power producti…
▽ More
We explore the use of deep reinforcement learning to provide strategies for long term scheduling of hydropower production. We consider a use-case where the aim is to optimise the yearly revenue given week-by-week inflows to the reservoir and electricity prices. The challenge is to decide between immediate water release at the spot price of electricity and storing the water for later power production at an unknown price, given constraints on the system. We successfully train a soft actor-critic algorithm on a simplified scenario with historical data from the Nordic power market. The presented model is not ready to substitute traditional optimisation tools but demonstrates the complementary potential of reinforcement learning in the data-rich field of hydropower scheduling.
△ Less
Submitted 9 December, 2020;
originally announced December 2020.
-
Improved VIV response prediction using adaptive parameters and data clustering
Authors:
Jie Wu,
Decao Yin,
Halvor Lie,
Signe Riemer-Sørensen,
Svein Sævik,
Michael Triantafyllou
Abstract:
Slender marine structures such as deep-water riser systems are continuously exposed to currents leading to vortex-induced vibrations (VIV) of the structure. This may result in amplified drag loads and fast accumulation of fatigue damage. Consequently, accurate prediction of VIV responses is of great importance for the safe design and operation of marine risers. Model tests with elastic pipes have…
▽ More
Slender marine structures such as deep-water riser systems are continuously exposed to currents leading to vortex-induced vibrations (VIV) of the structure. This may result in amplified drag loads and fast accumulation of fatigue damage. Consequently, accurate prediction of VIV responses is of great importance for the safe design and operation of marine risers. Model tests with elastic pipes have shown that VIV responses are influenced by many structural and hydrodynamic parameters, which have not been fully modelled in present frequency domain VIV prediction tools. Traditionally, predictions have been computed using a single set of hydrodynamic parameters, often leading to inconsistent prediction accuracy when compared with observed field measurements and experimental data. Hence, it is necessary to implement a high safety factor of 10 - 20 in the riser design, which increases development cost and adds extra constraints in the field operation. One way to compensate for the simplifications in the mathematical prediction model is to apply adaptive parameters to describe different riser responses. The objective of this work is to demonstrate a new method to improve the prediction consistency and accuracy by applying adaptive hydrodynamic parameters. In the present work, a four-step approach has been proposed: First, the measured VIV response will be analysed to identify key parameters to represent the response characteristics. These parameters will be grouped using data clustering algorithms. Secondly, optimal hydrodynamic parameters will be identified for each data group by optimisation against measured data. Thirdly, the VIV response using the obtained parameters will be calculated and the prediction accuracy evaluated. The correct hydrodynamic parameters to be used for new cases can be obtained from the clustering. This concept has been demonstrated with examples from experimental data.
△ Less
Submitted 10 March, 2020;
originally announced March 2020.
-
Data-driven prediction of vortex-induced vibration response of marine risers subjected to three-dimensional current
Authors:
Signe Riemer-Sørensen,
Jie Wu,
Halvor Lie,
Svein Sævik,
Sang-Woo Kim
Abstract:
Slender marine structures such as deep-water marine risers are subjected to currents and will normally experience Vortex Induced Vibrations (VIV), which can cause fast accumulation of fatigue damage. The ocean current is often three-dimensional (3D), i.e., the direction and magnitude of the current vary throughout the water column. Today, semi-empirical tools are used by the industry to predict VI…
▽ More
Slender marine structures such as deep-water marine risers are subjected to currents and will normally experience Vortex Induced Vibrations (VIV), which can cause fast accumulation of fatigue damage. The ocean current is often three-dimensional (3D), i.e., the direction and magnitude of the current vary throughout the water column. Today, semi-empirical tools are used by the industry to predict VIV induced fatigue on risers. The load model and hydrodynamic parameters in present VIV prediction tools are developed based on two-dimensional (2D) flow conditions, as it is challenging to consider the effect of 3D flow along the risers. Accordingly, the current profiles must be purposely made 2D during the design process, which leads to significant uncertainty in the prediction results. Further, due to the limitations in the laboratory, VIV model tests are mostly carried out under 2D flow conditions and thus little experimental data exist to document VIV response of riser subjected to varying directions of the current. However, a few experiments have been conducted with 3D current. We have used results from one of these experiments to investigate how well 1) traditional and 2) an alternative method based on a data driven prediction can describe VIV in 3D currents. Data driven modelling is particularly suited for complicated problems with many parameters and non-linear relationships. We have applied a data clustering algorithm to the experimental 3D flow data in order to identify measurable parameters that can influence responses. The riser responses are grouped based on their statistical characteristics, which relate to the direction of the flow. Furthermore we fit a random forest regression model to the measured VIV response and compare its performance with the predictions of existing VIV prediction tools (VIVANA-FD).
△ Less
Submitted 24 June, 2019;
originally announced June 2019.
