-
Hybrid Gaussian Process Modeling Applied to Economic Stochastic Model Predictive Control of Batch Processes
Authors:
E. Bradford,
L. Imsland,
M. Reble,
E. A. del Rio-Chanona
Abstract:
Nonlinear model predictive control (NMPC) is an efficient approach for the control of nonlinear multivariable dynamic systems with constraints, which however requires an accurate plant model. Plant models can often be determined from first principles, parts of the model are however difficult to derive using physical laws alone. In this paper a hybrid Gaussian process (GP) first principles modeling…
▽ More
Nonlinear model predictive control (NMPC) is an efficient approach for the control of nonlinear multivariable dynamic systems with constraints, which however requires an accurate plant model. Plant models can often be determined from first principles, parts of the model are however difficult to derive using physical laws alone. In this paper a hybrid Gaussian process (GP) first principles modeling scheme is proposed to overcome this issue, which exploits GPs to model the parts of the dynamic system that are difficult to describe using first principles. GPs not only give accurate predictions, but also quantify the residual uncertainty of this model. It is vital to account for this uncertainty in the control algorithm, to prevent constraint violations and performance deterioration. Monte Carlo samples of the GPs are generated offline to tighten constraints of the NMPC to ensure joint probabilistic constraint satisfaction online. Advantages of our method include fast online evaluation times, possibility to account for online learning alleviating conservativeness, and exploiting the flexibility of GPs and the data efficiency of first principle models. The algorithm is verified on a case study involving a challenging semi-batch bioreactor.
△ Less
Submitted 13 August, 2021;
originally announced August 2021.
-
Combining Gaussian processes and polynomial chaos expansions for stochastic nonlinear model predictive control
Authors:
E. Bradford,
L. Imsland
Abstract:
Model predictive control is an advanced control approach for multivariable systems with constraints, which is reliant on an accurate dynamic model. Most real dynamic models are however affected by uncertainties, which can lead to closed-loop performance deterioration and constraint violations. In this paper we introduce a new algorithm to explicitly consider time-invariant stochastic uncertainties…
▽ More
Model predictive control is an advanced control approach for multivariable systems with constraints, which is reliant on an accurate dynamic model. Most real dynamic models are however affected by uncertainties, which can lead to closed-loop performance deterioration and constraint violations. In this paper we introduce a new algorithm to explicitly consider time-invariant stochastic uncertainties in optimal control problems. The difficulty of propagating stochastic variables through nonlinear functions is dealt with by combining Gaussian processes with polynomial chaos expansions. The main novelty in this paper is to use this combination in an efficient fashion to obtain mean and variance estimates of nonlinear transformations. Using this algorithm, it is shown how to formulate both chance-constraints and a probabilistic objective for the optimal control problem. On a batch reactor case study we firstly verify the ability of the new approach to accurately approximate the probability distributions required. Secondly, a tractable stochastic nonlinear model predictive control approach is formulated with an economic objective to demonstrate the closed-loop performance of the method via Monte Carlo simulations.
△ Less
Submitted 9 March, 2021;
originally announced March 2021.
-
Real-Time Optimization Meets Bayesian Optimization and Derivative-Free Optimization: A Tale of Modifier Adaptation
Authors:
Ehecatl Antonio del Rio-Chanona,
Panagiotis Petsagkourakis,
Eric Bradford,
Jose Eduardo Alves Graciano,
Benoit Chachuat
Abstract:
This paper investigates a new class of modifier-adaptation schemes to overcome plant-model mismatch in real-time optimization of uncertain processes. The main contribution lies in the integration of concepts from the areas of Bayesian optimization and derivative-free optimization. The proposed schemes embed a physical model and rely on trust-region ideas to minimize risk during the exploration, wh…
▽ More
This paper investigates a new class of modifier-adaptation schemes to overcome plant-model mismatch in real-time optimization of uncertain processes. The main contribution lies in the integration of concepts from the areas of Bayesian optimization and derivative-free optimization. The proposed schemes embed a physical model and rely on trust-region ideas to minimize risk during the exploration, while employing Gaussian process regression to capture the plant-model mismatch in a non-parametric way and drive the exploration by means of acquisition functions. The benefits of using an acquisition function, knowing the process noise level, or specifying a nominal process model are illustrated on numerical case studies, including a semi-batch photobioreactor optimization problem.
△ Less
Submitted 1 February, 2021; v1 submitted 18 September, 2020;
originally announced September 2020.
-
Constrained Reinforcement Learning for Dynamic Optimization under Uncertainty
Authors:
Panagiotis Petsagkourakis,
Ilya Orson Sandoval,
Eric Bradford,
Dongda Zhang,
Ehecatl Antonio del Río Chanona
Abstract:
Dynamic real-time optimization (DRTO) is a challenging task due to the fact that optimal operating conditions must be computed in real time. The main bottleneck in the industrial application of DRTO is the presence of uncertainty. Many stochastic systems present the following obstacles: 1) plant-model mismatch, 2) process disturbances, 3) risks in violation of process constraints. To accommodate t…
▽ More
Dynamic real-time optimization (DRTO) is a challenging task due to the fact that optimal operating conditions must be computed in real time. The main bottleneck in the industrial application of DRTO is the presence of uncertainty. Many stochastic systems present the following obstacles: 1) plant-model mismatch, 2) process disturbances, 3) risks in violation of process constraints. To accommodate these difficulties, we present a constrained reinforcement learning (RL) based approach. RL naturally handles the process uncertainty by computing an optimal feedback policy. However, no state constraints can be introduced intuitively. To address this problem, we present a chance-constrained RL methodology. We use chance constraints to guarantee the probabilistic satisfaction of process constraints, which is accomplished by introducing backoffs, such that the optimal policy and backoffs are computed simultaneously. Backoffs are adjusted using the empirical cumulative distribution function to guarantee the satisfaction of a joint chance constraint. The advantage and performance of this strategy are illustrated through a stochastic dynamic bioprocess optimization problem, to produce sustainable high-value bioproducts.
△ Less
Submitted 4 June, 2020;
originally announced June 2020.
-
Stochastic data-driven model predictive control using Gaussian processes
Authors:
E. Bradford,
L. Imsland,
D. Zhang,
E. A. del Rio-Chanona
Abstract:
Nonlinear model predictive control (NMPC) is one of the few control methods that can handle multivariable nonlinear controlsystems with constraints. Gaussian processes (GPs) present a powerful tool to identify the required plant model and quantifythe residual uncertainty of the plant-model mismatch. It is crucial to consider this uncertainty, since it may lead to worsecontrol performance and const…
▽ More
Nonlinear model predictive control (NMPC) is one of the few control methods that can handle multivariable nonlinear controlsystems with constraints. Gaussian processes (GPs) present a powerful tool to identify the required plant model and quantifythe residual uncertainty of the plant-model mismatch. It is crucial to consider this uncertainty, since it may lead to worsecontrol performance and constraint violations. In this paper we propose a new method to design a GP-based NMPC algorithmfor finite horizon control problems. The method generates Monte Carlo samples of the GP offline for constraint tighteningusing back-offs. The tightened constraints then guarantee the satisfaction of chance constraints online. Advantages of our proposed approach over existing methods include fast online evaluation, consideration of closed-loop behaviour, and thepossibility to alleviate conservativeness by considering both online learning and state dependency of the uncertainty. The algorithm is verified on a challenging semi-batch bioprocess case study.
△ Less
Submitted 24 May, 2020; v1 submitted 5 August, 2019;
originally announced August 2019.
-
Reinforcement Learning for Batch Bioprocess Optimization
Authors:
Panagiotis Petsagkourakis,
Ilya Orson Sandoval,
Eric Bradford,
Dongda Zhang,
Ehecatl Antonio del Rio Chanona
Abstract:
Bioprocesses have received a lot of attention to produce clean and sustainable alternatives to fossil-based materials. However, they are generally difficult to optimize due to their unsteady-state operation modes and stochastic behaviours. Furthermore, biological systems are highly complex, therefore plant-model mismatch is often present. To address the aforementioned challenges we propose a Reinf…
▽ More
Bioprocesses have received a lot of attention to produce clean and sustainable alternatives to fossil-based materials. However, they are generally difficult to optimize due to their unsteady-state operation modes and stochastic behaviours. Furthermore, biological systems are highly complex, therefore plant-model mismatch is often present. To address the aforementioned challenges we propose a Reinforcement learning based optimization strategy for batch processes.
In this work, we applied the Policy Gradient method from batch-to-batch to update a control policy parametrized by a recurrent neural network. We assume that a preliminary process model is available, which is exploited to obtain a preliminary optimal control policy. Subsequently, this policy is updatedbased on measurements from thetrueplant. The capabilities of our proposed approach were tested on three case studies (one of which is nonsmooth) using a more complex process model for thetruesystemembedded with adequate process disturbance. Lastly, we discussed the advantages and disadvantages of this strategy compared against current existing approaches such as nonlinear model predictive control.
△ Less
Submitted 27 September, 2019; v1 submitted 15 April, 2019;
originally announced April 2019.
-
Stochastic Nonlinear Model Predictive Control with State Estimation by Incorporation of the Unscented Kalman Filter
Authors:
Eric Bradford,
Lars Imsland
Abstract:
Nonlinear model predictive control has become a popular approach to deal with highly nonlinear and unsteady state systems, the performance of which can however deteriorate due to unaccounted uncertainties. Model predictive control is commonly used with states from a state estimator in place of the exact states without consideration of the error. In this paper an approach is proposed by incorporati…
▽ More
Nonlinear model predictive control has become a popular approach to deal with highly nonlinear and unsteady state systems, the performance of which can however deteriorate due to unaccounted uncertainties. Model predictive control is commonly used with states from a state estimator in place of the exact states without consideration of the error. In this paper an approach is proposed by incorporating the unscented Kalman filter into the NMPC problem, which propagates uncertainty introduced from both the state estimate and additive noise from disturbances forward in time. The feasibility is maintained through probabilistic constraints based on the Gaussian approximations of the state distributions. The concept of robust horizon is introduced to limit the open loop covariances, which otherwise grow too large and lead to conservativeness and infeasibility of the MPC problem. The effectiveness of the approach was tested on a challenging semi batch reactor case study with an economic objective.
△ Less
Submitted 4 September, 2017;
originally announced September 2017.
