-
Incorporating a Deep Neural Network into Moving Horizon Estimation for Embedded Thermal Torque Derating of an Electric Machine
Authors:
Alexander Winkler,
Pranav Shah,
Katrin Baumgärtner,
Vasu Sharma,
David Gordon,
Jakob Andert
Abstract:
This study introduces a novel state estimation framework that incorporates Deep Neural Networks (DNNs) into Moving Horizon Estimation (MHE), shifting from traditional physics-based models to rapidly developed data-driven techniques. A DNN model with Long Short-Term Memory (LSTM) nodes is trained on synthetic data generated by a high-fidelity thermal model of a Permanent Magnet Synchronous Machine…
▽ More
This study introduces a novel state estimation framework that incorporates Deep Neural Networks (DNNs) into Moving Horizon Estimation (MHE), shifting from traditional physics-based models to rapidly developed data-driven techniques. A DNN model with Long Short-Term Memory (LSTM) nodes is trained on synthetic data generated by a high-fidelity thermal model of a Permanent Magnet Synchronous Machine (PMSM), which undergoes thermal derating as part of the torque control strategy in a battery electric vehicle. The MHE is constructed by integrating the trained DNN with a simplified driving dynamics model in a discrete-time formulation, incorporating the LSTM hidden and cell states in the state vector to retain system dynamics. The resulting optimal control problem (OCP) is formulated as a nonlinear program (NLP) and implemented using the acados framework. Model-in-the-loop (MiL) simulations demonstrate accurate temperature estimation, even under noisy sensor conditions or failures. Achieving threefold real-time capability on embedded hardware confirms the feasibility of the approach for practical deployment. The primary focus of this study is to assess the feasibility of the MHE framework using a DNN-based plant model instead of focusing on quantitative comparisons of vehicle performance. Overall, this research highlights the potential of DNN-based MHE for real-time, safety-critical applications by combining the strengths of model-based and data-driven methods.
△ Less
Submitted 17 April, 2025;
originally announced April 2025.
-
Safe Reinforcement Learning-based Control for Hydrogen Diesel Dual-Fuel Engines
Authors:
Vasu Sharma,
Alexander Winkler,
Armin Norouzi,
Jakob Andert,
David Gordon,
Hongsheng Guo
Abstract:
The urgent energy transition requirements towards a sustainable future stretch across various industries and are a significant challenge facing humanity. Hydrogen promises a clean, carbon-free future, with the opportunity to integrate with existing solutions in the transportation sector. However, adding hydrogen to existing technologies such as diesel engines requires additional modeling effort. R…
▽ More
The urgent energy transition requirements towards a sustainable future stretch across various industries and are a significant challenge facing humanity. Hydrogen promises a clean, carbon-free future, with the opportunity to integrate with existing solutions in the transportation sector. However, adding hydrogen to existing technologies such as diesel engines requires additional modeling effort. Reinforcement Learning (RL) enables interactive data-driven learning that eliminates the need for mathematical modeling. The algorithms, however, may not be real-time capable and need large amounts of data to work in practice. This paper presents a novel approach which uses offline model learning with RL to demonstrate safe control of a 4.5 L Hydrogen Diesel Dual-Fuel (H2DF) engine. The controllers are demonstrated to be constraint compliant and can leverage a novel state-augmentation approach for sample-efficient learning. The offline policy is subsequently experimentally validated on the real engine where the control algorithm is executed on a Raspberry Pi controller and requires 6 times less computation time compared to online Model Predictive Control (MPC) optimization.
△ Less
Submitted 13 February, 2025;
originally announced February 2025.
-
Introducing a Deep Neural Network-based Model Predictive Control Framework for Rapid Controller Implementation
Authors:
David C. Gordon,
Alexander Winkler,
Julian Bedei,
Patrick Schaber,
Jakob Andert,
Charles R. Koch
Abstract:
Model Predictive Control (MPC) provides an optimal control solution based on a cost function while allowing for the implementation of process constraints. As a model-based optimal control technique, the performance of MPC strongly depends on the model used where a trade-off between model computation time and prediction performance exists. One solution is the integration of MPC with a machine learn…
▽ More
Model Predictive Control (MPC) provides an optimal control solution based on a cost function while allowing for the implementation of process constraints. As a model-based optimal control technique, the performance of MPC strongly depends on the model used where a trade-off between model computation time and prediction performance exists. One solution is the integration of MPC with a machine learning (ML) based process model which are quick to evaluate online. This work presents the experimental implementation of a deep neural network (DNN) based nonlinear MPC for Homogeneous Charge Compression Ignition (HCCI) combustion control. The DNN model consists of a Long Short-Term Memory (LSTM) network surrounded by fully connected layers which was trained using experimental engine data and showed acceptable prediction performance with under 5% error for all outputs. Using this model, the MPC is designed to track the Indicated Mean Effective Pressure (IMEP) and combustion phasing trajectories, while minimizing several parameters. Using the acados software package to enable the real-time implementation of the MPC on an ARM Cortex A72, the optimization calculations are completed within 1.4 ms. The external A72 processor is integrated with the prototyping engine controller using a UDP connection allowing for rapid experimental deployment of the NMPC. The IMEP trajectory following of the developed controller was excellent, with a root-mean-square error of 0.133 bar, in addition to observing process constraints.
△ Less
Submitted 12 October, 2023;
originally announced October 2023.
-
Machine Learning Integrated with Model Predictive Control for Imitative Optimal Control of Compression Ignition Engines
Authors:
Armin Norouzi,
Saeid Shahpouri,
David Gordon,
Alexander Winkler,
Eugen Nuss,
Dirk Abel,
Jakob Andert,
Mahdi Shahbakhti,
Charles Robert Koch
Abstract:
The high thermal efficiency and reliability of the compression-ignition engine makes it the first choice for many applications. For this to continue, a reduction of the pollutant emissions is needed. One solution is the use of machine learning (ML) and model predictive control (MPC) to minimize emissions and fuel consumption, without adding substantial computational cost to the engine controller.…
▽ More
The high thermal efficiency and reliability of the compression-ignition engine makes it the first choice for many applications. For this to continue, a reduction of the pollutant emissions is needed. One solution is the use of machine learning (ML) and model predictive control (MPC) to minimize emissions and fuel consumption, without adding substantial computational cost to the engine controller. ML is developed in this paper for both modeling engine performance and emissions and for imitating the behaviour of an Linear Parameter Varying (LPV) MPC. Using a support vector machine-based linear parameter varying model of the engine performance and emissions, a model predictive controller is implemented for a 4.5 Cummins diesel engine. This online optimized MPC solution offers advantages in minimizing the \nox~emissions and fuel consumption compared to the baseline feedforward production controller. To reduce the computational cost of this MPC, a deep learning scheme is designed to mimic the behavior of the developed controller. The performance in reducing NOx emissions at a constant load by the imitative controller is similar to that of the online optimized MPC compared to the Cummins production controller. In addition, the imitative controller requires 50 times less computation time compared to that of the online MPC optimization.
△ Less
Submitted 2 August, 2022; v1 submitted 31 March, 2022;
originally announced April 2022.
-
Deep Learning based Model Predictive Control for Compression Ignition Engines
Authors:
Armin Norouzi,
Saeid Shahpouri,
David Gordon,
Alexander Winkler,
Eugen Nuss,
Dirk Abel,
Jakob Andert,
Mahdi Shahbakhti,
Charles Robert Koch
Abstract:
Machine learning (ML) and a nonlinear model predictive controller (NMPC) are used in this paper to minimize the emissions and fuel consumption of a compression ignition engine. In this work machine learning is applied in two methods. In the first application, ML is used to identify a model for implementation in model predictive control optimization problems. In the second application, ML is used a…
▽ More
Machine learning (ML) and a nonlinear model predictive controller (NMPC) are used in this paper to minimize the emissions and fuel consumption of a compression ignition engine. In this work machine learning is applied in two methods. In the first application, ML is used to identify a model for implementation in model predictive control optimization problems. In the second application, ML is used as a replacement of the NMPC where the ML controller learns the optimal control action by imitating or mimicking the behavior of the model predictive controller. In this study, a deep recurrent neural network including long-short term memory (LSTM) layers are used to model the emissions and performance of an industrial 4.5 liter 4-cylinder Cummins diesel engine. This model is then used for model predictive controller implementation. Then, a deep learning scheme is deployed to clone the behavior of the developed controller. In the LSTM integration, a novel scheme is used by augmenting hidden and cell states of the network in an NMPC optimization problem. The developed LSTM-NMPC and the imitative NMPC are compared with the Cummins calibrated Engine Control Unit (ECU) model in an experimentally validated engine simulation platform. Results show a significant reduction in Nitrogen Oxides (\nox) emissions and a slight decrease in the injected fuel quantity while maintaining the same load. In addition, the imitative NMPC has a similar performance as the NMPC but with a two orders of magnitude reduction of the computation time.
△ Less
Submitted 4 August, 2022; v1 submitted 31 March, 2022;
originally announced April 2022.
-
MRSaiFE: Tissue Heating Prediction for MRI: a Feasibility Study
Authors:
Simone Angela Winkler,
Isabelle Saniour,
Akshay Chaudhari,
Fraser Robb,
J Thomas Vaughan
Abstract:
A to-date unsolved and highly limiting safety concern for Ultra High-Field (UHF) magnetic resonance imaging (MRI) is the deposition of radiofrequency (RF) power in the body, quantified by the specific absorption rate (SAR), leading to dangerous tissue heating/damage in the form of local SAR hotspots that cannot currently be measured/monitored, thereby severely limiting the applicability of the tec…
▽ More
A to-date unsolved and highly limiting safety concern for Ultra High-Field (UHF) magnetic resonance imaging (MRI) is the deposition of radiofrequency (RF) power in the body, quantified by the specific absorption rate (SAR), leading to dangerous tissue heating/damage in the form of local SAR hotspots that cannot currently be measured/monitored, thereby severely limiting the applicability of the technology for clinical practice and in regulatory approval. The goal of this study has been to show proof of concept of an artificial intelligence (AI) based exam-integrated real-time MRI safety prediction software (MRSaiFE) facilitating the safe generation of 3T and 7T images by means of accurate local SAR-monitoring at sub-W/kg levels. We trained the software with a small database of image as a feasibility study and achieved successful proof of concept for both field strengths. SAR patterns were predicted with a residual root mean squared error (RSME) of <11% along with a structural similarity (SSIM) level of >84% for both field strengths (3T and 7T).
△ Less
Submitted 1 February, 2021;
originally announced February 2021.
-
Optimal and Learning Control for Autonomous Robots
Authors:
Jonas Buchli,
Farbod Farshidian,
Alexander Winkler,
Timothy Sandy,
Markus Giftthaler
Abstract:
Optimal and Learning Control for Autonomous Robots has been taught in the Robotics, Systems and Controls Masters at ETH Zurich with the aim to teach optimal control and reinforcement learning for closed loop control problems from a unified point of view. The starting point is the formulation of of an optimal control problem and deriving the different types of solutions and algorithms from there. T…
▽ More
Optimal and Learning Control for Autonomous Robots has been taught in the Robotics, Systems and Controls Masters at ETH Zurich with the aim to teach optimal control and reinforcement learning for closed loop control problems from a unified point of view. The starting point is the formulation of of an optimal control problem and deriving the different types of solutions and algorithms from there. These lecture notes aim at supporting this unified view with a unified notation wherever possible, and a bit of a translation help to compare the terminology and notation in the different fields. The course assumes basic knowledge of Control Theory, Linear Algebra and Stochastic Calculus.
△ Less
Submitted 30 August, 2017;
originally announced August 2017.
-
An Efficient Optimal Planning and Control Framework For Quadrupedal Locomotion
Authors:
Farbod Farshidian,
Michael Neunert,
Alexander W. Winkler,
Gonzalo Rey,
Jonas Buchli
Abstract:
In this paper, we present an efficient Dynamic Programing framework for optimal planning and control of legged robots. First we formulate this problem as an optimal control problem for switched systems. Then we propose a multi--level optimization approach to find the optimal switching times and the optimal continuous control inputs. Through this scheme, the decomposed optimization can potentially…
▽ More
In this paper, we present an efficient Dynamic Programing framework for optimal planning and control of legged robots. First we formulate this problem as an optimal control problem for switched systems. Then we propose a multi--level optimization approach to find the optimal switching times and the optimal continuous control inputs. Through this scheme, the decomposed optimization can potentially be done more efficiently than the combined approach. Finally, we present a continuous-time constrained LQR algorithm which simultaneously optimizes the feedforward and feedback controller with $O(n)$ time-complexity. In order to validate our approach, we show the performance of our framework on a quadrupedal robot. We choose the Center of Mass dynamics and the full kinematic formulation as the switched system model where the switching times as well as the contact forces and the joint velocities are optimized for different locomotion tasks such as gap crossing, walking and trotting.
△ Less
Submitted 4 March, 2017; v1 submitted 30 September, 2016;
originally announced September 2016.
