-
A deep latent variable model for semi-supervised multi-unit soft sensing in industrial processes
Authors:
Bjarne Grimstad,
Kristian Løvland,
Lars S. Imsland,
Vidar Gunnerud
Abstract:
In many industrial processes, an apparent lack of data limits the development of data-driven soft sensors. There are, however, often opportunities to learn stronger models by being more data-efficient. To achieve this, one can leverage knowledge about the data from which the soft sensor is learned. Taking advantage of properties frequently possessed by industrial data, we introduce a deep latent v…
▽ More
In many industrial processes, an apparent lack of data limits the development of data-driven soft sensors. There are, however, often opportunities to learn stronger models by being more data-efficient. To achieve this, one can leverage knowledge about the data from which the soft sensor is learned. Taking advantage of properties frequently possessed by industrial data, we introduce a deep latent variable model for semi-supervised multi-unit soft sensing. This hierarchical, generative model is able to jointly model different units, as well as learning from both labeled and unlabeled data.
An empirical study of multi-unit soft sensing is conducted using two datasets: a synthetic dataset of single-phase fluid flow, and a large, real dataset of multi-phase flow in oil and gas wells. We show that by combining semi-supervised and multi-task learning, the proposed model achieves superior results, outperforming current leading methods for this soft sensing problem. We also show that when a model has been trained on a multi-unit dataset, it may be finetuned to previously unseen units using only a handful of data points. In this finetuning procedure, unlabeled data improve soft sensor performance; remarkably, this is true even when no labeled data are available.
△ Less
Submitted 18 July, 2024;
originally announced July 2024.
-
Multi-task and few-shot learning in virtual flow metering
Authors:
Kristian Løvland,
Bjarne Grimstad,
Lars S. Imsland
Abstract:
Recent literature has explored various ways to improve soft sensors by utilizing learning algorithms with transferability. A performance gain is generally attained when knowledge is transferred among strongly related soft sensor learning tasks. One setting where it is reasonable to expect strongly related tasks, is when learning soft sensors for separate process units that are of the same type. Ap…
▽ More
Recent literature has explored various ways to improve soft sensors by utilizing learning algorithms with transferability. A performance gain is generally attained when knowledge is transferred among strongly related soft sensor learning tasks. One setting where it is reasonable to expect strongly related tasks, is when learning soft sensors for separate process units that are of the same type. Applying methods that exploit transferability in this setting leads to what we call multi-unit soft sensing.
This paper formulates a probabilistic, hierarchical model for multi-unit soft sensing. The model is implemented using a deep neural network. The proposed learning method is studied empirically on a large-scale industrial case by developing virtual flow meters (a type of soft sensor) for 80 petroleum wells. We investigate how the model generalizes with the number of wells/units. We demonstrate that multi-unit models learned from data from many wells permit few-shot learning of virtual flow meters for new wells. Surprisingly, regarding the difficulty of the tasks, few-shot learning on 1-3 data points often leads to high performance on new wells.
△ Less
Submitted 16 September, 2025; v1 submitted 27 September, 2023;
originally announced September 2023.
-
Sequential Monte Carlo applied to virtual flow meter calibration
Authors:
Anders T. Sandnes,
Bjarne Grimstad,
Odd Kolbjørnsen
Abstract:
Soft-sensors are gaining popularity due to their ability to provide estimates of key process variables with little intervention required on the asset and at a low cost. In oil and gas production, virtual flow metering (VFM) is a popular soft-sensor that attempts to estimate multiphase flow rates in real time. VFMs are based on models, and these models require calibration. The calibration is highly…
▽ More
Soft-sensors are gaining popularity due to their ability to provide estimates of key process variables with little intervention required on the asset and at a low cost. In oil and gas production, virtual flow metering (VFM) is a popular soft-sensor that attempts to estimate multiphase flow rates in real time. VFMs are based on models, and these models require calibration. The calibration is highly dependent on the application, both due to the great diversity of the models, and in the available measurements. The most accurate calibration is achieved by careful tuning of the VFM parameters to well tests, but this can be work intensive, and not all wells have frequent well test data available. This paper presents a calibration method based on the measurement provided by the production separator, and the assumption that the observed flow should be equal to the sum of flow rates from each individual well. This allows us to jointly calibrate the VFMs continuously. The method applies Sequential Monte Carlo (SMC) to infer a tuning factor and the flow composition for each well. The method is tested on a case with ten wells, using both synthetic and real data. The results are promising and the method is able to provide reasonable estimates of the parameters without relying on well tests. However, some challenges are identified and discussed, particularly related to the process noise and how to manage varying data quality.
△ Less
Submitted 13 April, 2023;
originally announced April 2023.
-
Multi-task neural networks by learned contextual inputs
Authors:
Anders T. Sandnes,
Bjarne Grimstad,
Odd Kolbjørnsen
Abstract:
This paper explores learned-context neural networks. It is a multi-task learning architecture based on a fully shared neural network and an augmented input vector containing trainable task parameters. The architecture is interesting due to its powerful task adaption mechanism, which facilitates a low-dimensional task parameter space. Theoretically, we show that a scalar task parameter is sufficien…
▽ More
This paper explores learned-context neural networks. It is a multi-task learning architecture based on a fully shared neural network and an augmented input vector containing trainable task parameters. The architecture is interesting due to its powerful task adaption mechanism, which facilitates a low-dimensional task parameter space. Theoretically, we show that a scalar task parameter is sufficient for universal approximation of all tasks, which is not necessarily the case for more common architectures. Empirically it is shown that, for homogeneous tasks, the dimension of the task parameter may vary with the complexity of the tasks, but a small task parameter space is generally viable. The task parameter space is found to be well-behaved, which simplifies workflows related to updating models as new data arrives, and learning new tasks with the shared parameters are frozen. Additionally, the architecture displays robustness towards datasets where tasks have few data points. The architecture's performance is compared to similar neural network architectures on ten datasets, with competitive results.
△ Less
Submitted 6 August, 2025; v1 submitted 1 March, 2023;
originally announced March 2023.
-
Adjustment formulas for learning causal steady-state models from closed-loop operational data
Authors:
Kristian Løvland,
Bjarne Grimstad,
Lars Struen Imsland
Abstract:
Steady-state models which have been learned from historical operational data may be unfit for model-based optimization unless correlations in the training data which are introduced by control are accounted for. Using recent results from work on structural dynamical causal models, we derive a formula for adjusting for this control confounding, enabling the estimation of a causal steady-state model…
▽ More
Steady-state models which have been learned from historical operational data may be unfit for model-based optimization unless correlations in the training data which are introduced by control are accounted for. Using recent results from work on structural dynamical causal models, we derive a formula for adjusting for this control confounding, enabling the estimation of a causal steady-state model from closed-loop steady-state data. The formula assumes that the available data have been gathered under some fixed control law. It works by estimating and taking into account the disturbance which the controller is trying to counteract, and enables learning from data gathered under both feedforward and feedback control.
△ Less
Submitted 10 November, 2022;
originally announced November 2022.
-
Passive learning to address nonstationarity in virtual flow metering applications
Authors:
Mathilde Hotvedt,
Bjarne Grimstad,
Lars Imsland
Abstract:
Steady-state process models are common in virtual flow meter applications due to low computational complexity, and low model development and maintenance cost. Nevertheless, the prediction performance of steady-state models typically degrades with time due to the inherent nonstationarity of the underlying process being modeled. Few studies have investigated how learning methods can be applied to su…
▽ More
Steady-state process models are common in virtual flow meter applications due to low computational complexity, and low model development and maintenance cost. Nevertheless, the prediction performance of steady-state models typically degrades with time due to the inherent nonstationarity of the underlying process being modeled. Few studies have investigated how learning methods can be applied to sustain the prediction accuracy of steady-state virtual flow meters. This paper explores passive learning, where the model is frequently calibrated to new data, as a way to address nonstationarity and improve long-term performance. An advantage with passive learning is that it is compatible with models used in the industry. Two passive learning methods, periodic batch learning and online learning, are applied with varying calibration frequency to train virtual flow meters. Six different model types, ranging from data-driven to first-principles, are trained on historical production data from 10 petroleum wells. The results are two-fold: first, in the presence of frequently arriving measurements, frequent model updating sustains an excellent prediction performance over time; second, in the presence of intermittent and infrequently arriving measurements, frequent updating in addition to the utilization of expert knowledge is essential to increase the performance accuracy. The investigation may be of interest to experts developing soft-sensors for nonstationary processes, such as virtual flow meters.
△ Less
Submitted 7 February, 2022;
originally announced February 2022.
-
When is gray-box modeling advantageous for virtual flow metering?
Authors:
M. Hotvedt,
B. Grimstad,
D. Ljungquist,
L. Imsland
Abstract:
Integration of physics and machine learning in virtual flow metering applications is known as gray-box modeling. The combination is believed to enhance multiphase flow rate predictions. However, the superiority of gray-box models is yet to be demonstrated in the literature. This article examines scenarios where a gray-box model is expected to outperform physics-based and data-driven models. The ex…
▽ More
Integration of physics and machine learning in virtual flow metering applications is known as gray-box modeling. The combination is believed to enhance multiphase flow rate predictions. However, the superiority of gray-box models is yet to be demonstrated in the literature. This article examines scenarios where a gray-box model is expected to outperform physics-based and data-driven models. The experiments are conducted with synthetic data where properties of the underlying data generating process are known and controlled. The results show that a gray-box model yields increased prediction accuracy over a physics-based model in the presence of process-model mismatch. They also show improvements over a data-driven model when the amount of available data is small. On the other hand, gray-box and data-driven models are similarly influenced by noisy measurements. Lastly, the results indicate that a gray-box approach may be advantageous in nonstationary process conditions. Unfortunately, choosing the best model prior to training is challenging, and overhead on model development is unavoidable.
△ Less
Submitted 11 October, 2021;
originally announced October 2021.
-
On gray-box modeling for virtual flow metering
Authors:
Mathilde Hotvedt,
Bjarne Grimstad,
Dag Ljungquist,
Lars Imsland
Abstract:
A virtual flow meter (VFM) enables continuous prediction of flow rates in petroleum production systems. The predicted flow rates may aid the daily control and optimization of a petroleum asset. Gray-box modeling is an approach that combines mechanistic and data-driven modeling. The objective is to create a computationally feasible VFM for use in real-time applications, with high prediction accurac…
▽ More
A virtual flow meter (VFM) enables continuous prediction of flow rates in petroleum production systems. The predicted flow rates may aid the daily control and optimization of a petroleum asset. Gray-box modeling is an approach that combines mechanistic and data-driven modeling. The objective is to create a computationally feasible VFM for use in real-time applications, with high prediction accuracy and scientifically consistent behavior. This article investigates five different gray-box model types in an industrial case study using real, historical production data from 10 petroleum wells, spanning at most four years of production. The results are diverse with an oil flow rate prediction error in the range of 1.8%-40.6%. Further, the study casts light upon the nontrivial task of balancing learning from both physics and data. Consequently, providing general recommendations towards the suitability of different hybrid models is challenging. Nevertheless, the results are promising and indicate that gray-box VFMs may reduce the prediction error of a mechanistic VFM while remaining scientifically consistent. The findings motivate further experimentation with gray-box VFM models and suggest several future research directions to improve upon the performance and scientific consistency.
△ Less
Submitted 27 October, 2021; v1 submitted 23 March, 2021;
originally announced March 2021.
-
Multi-task learning for virtual flow metering
Authors:
Anders T. Sandnes,
Bjarne Grimstad,
Odd Kolbjørnsen
Abstract:
Virtual flow metering (VFM) is a cost-effective and non-intrusive technology for inferring multiphase flow rates in petroleum assets. Inferences about flow rates are fundamental to decision support systems that operators extensively rely on. Data-driven VFM, where mechanistic models are replaced with machine learning models, has recently gained attention due to its promise of lower maintenance cos…
▽ More
Virtual flow metering (VFM) is a cost-effective and non-intrusive technology for inferring multiphase flow rates in petroleum assets. Inferences about flow rates are fundamental to decision support systems that operators extensively rely on. Data-driven VFM, where mechanistic models are replaced with machine learning models, has recently gained attention due to its promise of lower maintenance costs. While excellent performances in small sample studies have been reported in the literature, there is still considerable doubt about the robustness of data-driven VFM. In this paper, we propose a new multi-task learning (MTL) architecture for data-driven VFM. Our method differs from previous methods in that it enables learning across oil and gas wells. We study the method by modeling 55 wells from four petroleum assets and compare the results with two single-task baseline models. Our findings show that MTL improves robustness over single-task methods, without sacrificing performance. MTL yields a 25-50% error reduction on average for the assets where single-task architectures are struggling.
△ Less
Submitted 10 September, 2021; v1 submitted 15 March, 2021;
originally announced March 2021.
-
Bayesian Neural Networks for Virtual Flow Metering: An Empirical Study
Authors:
Bjarne Grimstad,
Mathilde Hotvedt,
Anders T. Sandnes,
Odd Kolbjørnsen,
Lars S. Imsland
Abstract:
Recent works have presented promising results from the application of machine learning (ML) to the modeling of flow rates in oil and gas wells. Encouraging results and advantageous properties of ML models, such as computationally cheap evaluation and ease of calibration to new data, have sparked optimism for the development of data-driven virtual flow meters (VFMs). Data-driven VFMs are developed…
▽ More
Recent works have presented promising results from the application of machine learning (ML) to the modeling of flow rates in oil and gas wells. Encouraging results and advantageous properties of ML models, such as computationally cheap evaluation and ease of calibration to new data, have sparked optimism for the development of data-driven virtual flow meters (VFMs). Data-driven VFMs are developed in the small data regime, where it is important to question the uncertainty and robustness of models. The modeling of uncertainty may help to build trust in models, which is a prerequisite for industrial applications. The contribution of this paper is the introduction of a probabilistic VFM based on Bayesian neural networks. Uncertainty in the model and measurements is described, and the paper shows how to perform approximate Bayesian inference using variational inference. The method is studied by modeling on a large and heterogeneous dataset, consisting of 60 wells across five different oil and gas assets. The predictive performance is analyzed on historical and future test data, where an average error of 4-6% and 8-13% is achieved for the 50% best performing models, respectively. Variational inference appears to provide more robust predictions than the reference approach on future data. Prediction performance and uncertainty calibration is explored in detail and discussed in light of four data challenges. The findings motivate the development of alternative strategies to improve the robustness of data-driven VFMs.
△ Less
Submitted 13 June, 2021; v1 submitted 2 February, 2021;
originally announced February 2021.
-
Developing a Hybrid Data-Driven, Mechanistic Virtual Flow Meter -- a Case Study
Authors:
Mathilde Hotvedt,
Bjarne Grimstad,
Lars Imsland
Abstract:
Virtual flow meters, mathematical models predicting production flow rates in petroleum assets, are useful aids in production monitoring and optimization. Mechanistic models based on first-principles are most common, however, data-driven models exploiting patterns in measurements are gaining popularity. This research investigates a hybrid modeling approach, utilizing techniques from both the aforem…
▽ More
Virtual flow meters, mathematical models predicting production flow rates in petroleum assets, are useful aids in production monitoring and optimization. Mechanistic models based on first-principles are most common, however, data-driven models exploiting patterns in measurements are gaining popularity. This research investigates a hybrid modeling approach, utilizing techniques from both the aforementioned areas of expertise, to model a well production choke. The choke is represented with a simplified set of first-principle equations and a neural network to estimate the valve flow coefficient. Historical production data from the petroleum platform Edvard Grieg is used for model validation. Additionally, a mechanistic and a data-driven model are constructed for comparison of performance. A practical framework for development of models with varying degree of hybridity and stochastic optimization of its parameters is established. Results of the hybrid model performance are promising albeit with considerable room for improvements.
△ Less
Submitted 26 October, 2020; v1 submitted 7 February, 2020;
originally announced February 2020.
-
ReLU Networks as Surrogate Models in Mixed-Integer Linear Programs
Authors:
Bjarne Grimstad,
Henrik Andersson
Abstract:
We consider the embedding of piecewise-linear deep neural networks (ReLU networks) as surrogate models in mixed-integer linear programming (MILP) problems. A MILP formulation of ReLU networks has recently been applied by many authors to probe for various model properties subject to input bounds. The formulation is obtained by programming each ReLU operator with a binary variable and applying the b…
▽ More
We consider the embedding of piecewise-linear deep neural networks (ReLU networks) as surrogate models in mixed-integer linear programming (MILP) problems. A MILP formulation of ReLU networks has recently been applied by many authors to probe for various model properties subject to input bounds. The formulation is obtained by programming each ReLU operator with a binary variable and applying the big-M method. The efficiency of the formulation hinges on the tightness of the bounds defined by the big-M values. When ReLU networks are embedded in a larger optimization problem, the presence of output bounds can be exploited in bound tightening. To this end, we devise and study several bound tightening procedures that consider both input and output bounds. Our numerical results show that bound tightening may reduce solution times considerably, and that small-sized ReLU networks are suitable as surrogate models in mixed-integer linear programs.
△ Less
Submitted 26 September, 2019; v1 submitted 6 July, 2019;
originally announced July 2019.
