Showing 1–29 of 29 results for author: Winkler, A

Hybrid Reinforcement Learning and Model Predictive Control for Adaptive Control of Hydrogen-Diesel Dual-Fuel Combustion

Authors: Julian Bedei, Murray McBain, Alexander Winkler, Charles Robert Koch, Jakob Andert, David Gordon

Abstract: Reinforcement Learning (RL) and Machine Learning Integrated Model Predictive Control (ML-MPC) are promising approaches for optimizing hydrogen-diesel dual-fuel engine control, as they can effectively control multiple-input multiple-output systems and nonlinear processes. ML-MPC is advantageous for providing safe and optimal controls, ensuring the engine operates within predefined safety limits. In… ▽ More Reinforcement Learning (RL) and Machine Learning Integrated Model Predictive Control (ML-MPC) are promising approaches for optimizing hydrogen-diesel dual-fuel engine control, as they can effectively control multiple-input multiple-output systems and nonlinear processes. ML-MPC is advantageous for providing safe and optimal controls, ensuring the engine operates within predefined safety limits. In contrast, RL is distinguished by its adaptability to changing conditions through its learning-based approach. However, the practical implementation of either method alone poses challenges. RL requires high variance in control inputs during early learning phases, which can pose risks to the system by potentially executing unsafe actions, leading to mechanical damage. Conversely, ML-MPC relies on an accurate system model to generate optimal control inputs and has limited adaptability to system drifts, such as injector aging, which naturally occur in engine applications. To address these limitations, this study proposes a hybrid RL and ML-MPC approach that uses an ML-MPC framework while incorporating an RL agent to dynamically adjust the ML-MPC load tracking reference in response to changes in the environment. At the same time, the ML-MPC ensures that actions stay safe throughout the RL agent's exploration. To evaluate the effectiveness of this approach, fuel pressure is deliberately varied to introduce a model-plant mismatch between the ML-MPC and the engine test bench. The result of this mismatch is a root mean square error (RMSE) in indicated mean effective pressure of 0.57 bar when running the ML-MPC. The experimental results demonstrate that RL successfully adapts to changing boundary conditions by altering the tracking reference while ML-MPC ensures safe control inputs. The quantitative improvement in load tracking by implementing RL is an RSME of 0.44 bar. △ Less

Submitted 6 May, 2025; v1 submitted 23 April, 2025; originally announced April 2025.

Safe Reinforcement Learning for Real-World Engine Control

Authors: Julian Bedei, Lucas Koch, Kevin Badalian, Alexander Winkler, Patrick Schaber, Jakob Andert

Abstract: This work introduces a toolchain for applying Reinforcement Learning (RL), specifically the Deep Deterministic Policy Gradient (DDPG) algorithm, in safety-critical real-world environments. As an exemplary application, transient load control is demonstrated on a single-cylinder internal combustion engine testbench in Homogeneous Charge Compression Ignition (HCCI) mode, that offers high thermal effi… ▽ More This work introduces a toolchain for applying Reinforcement Learning (RL), specifically the Deep Deterministic Policy Gradient (DDPG) algorithm, in safety-critical real-world environments. As an exemplary application, transient load control is demonstrated on a single-cylinder internal combustion engine testbench in Homogeneous Charge Compression Ignition (HCCI) mode, that offers high thermal efficiency and low emissions. However, HCCI poses challenges for traditional control methods due to its nonlinear, autoregressive, and stochastic nature. RL provides a viable solution, however, safety concerns, such as excessive pressure rise rates, must be addressed when applying to HCCI. A single unsuitable control input can severely damage the engine or cause misfiring and shut down. Additionally, operating limits are not known a priori and must be determined experimentally. To mitigate these risks, real-time safety monitoring based on the k-nearest neighbor algorithm is implemented, enabling safe interaction with the testbench. The feasibility of this approach is demonstrated as the RL agent learns a control policy through interaction with the testbench. A root mean square error of 0.1374 bar is achieved for the indicated mean effective pressure, comparable to neural network-based controllers from the literature. The toolchain's flexibility is further demonstrated by adapting the agent's policy to increase ethanol energy shares, promoting renewable fuel use while maintaining safety. This RL approach addresses the longstanding challenge of applying RL to safety-critical real-world environments. The developed toolchain, with its adaptability and safety mechanisms, paves the way for future applicability of RL in engine testbenches and other safety-critical settings. △ Less

Submitted 27 January, 2025; originally announced January 2025.

Atmospheric Transport Modeling of CO$_2$ with Neural Networks

Authors: Vitus Benson, Ana Bastos, Christian Reimers, Alexander J. Winkler, Fanny Yang, Markus Reichstein

Abstract: Accurately describing the distribution of CO$_2$ in the atmosphere with atmospheric tracer transport models is essential for greenhouse gas monitoring and verification support systems to aid implementation of international climate agreements. Large deep neural networks are poised to revolutionize weather prediction, which requires 3D modeling of the atmosphere. While similar in this regard, atmosp… ▽ More Accurately describing the distribution of CO$_2$ in the atmosphere with atmospheric tracer transport models is essential for greenhouse gas monitoring and verification support systems to aid implementation of international climate agreements. Large deep neural networks are poised to revolutionize weather prediction, which requires 3D modeling of the atmosphere. While similar in this regard, atmospheric transport modeling is subject to new challenges. Both, stable predictions for longer time horizons and mass conservation throughout need to be achieved, while IO plays a larger role compared to computational costs. In this study we explore four different deep neural networks (UNet, GraphCast, Spherical Fourier Neural Operator and SwinTransformer) which have proven as state-of-the-art in weather prediction to assess their usefulness for atmospheric tracer transport modeling. For this, we assemble the CarbonBench dataset, a systematic benchmark tailored for machine learning emulators of Eulerian atmospheric transport. Through architectural adjustments, we decouple the performance of our emulators from the distribution shift caused by a steady rise in atmospheric CO$_2$. More specifically, we center CO$_2$ input fields to zero mean and then use an explicit flux scheme and a mass fixer to assure mass balance. This design enables stable and mass conserving transport for over 6 months with all four neural network architectures. In our study, the SwinTransformer displays particularly strong emulation skill (90-day $R^2 > 0.99$), with physically plausible emulation even for forward runs of multiple years. This work paves the way forward towards high resolution forward and inverse modeling of inert trace gases with neural networks. △ Less

Submitted 20 August, 2024; originally announced August 2024.

Comments: Code: https://github.com/vitusbenson/carbonbench

Omnigrasp: Grasping Diverse Objects with Simulated Humanoids

Authors: Zhengyi Luo, Jinkun Cao, Sammy Christen, Alexander Winkler, Kris Kitani, Weipeng Xu

Abstract: We present a method for controlling a simulated humanoid to grasp an object and move it to follow an object's trajectory. Due to the challenges in controlling a humanoid with dexterous hands, prior methods often use a disembodied hand and only consider vertical lifts or short trajectories. This limited scope hampers their applicability for object manipulation required for animation and simulation.… ▽ More We present a method for controlling a simulated humanoid to grasp an object and move it to follow an object's trajectory. Due to the challenges in controlling a humanoid with dexterous hands, prior methods often use a disembodied hand and only consider vertical lifts or short trajectories. This limited scope hampers their applicability for object manipulation required for animation and simulation. To close this gap, we learn a controller that can pick up a large number (>1200) of objects and carry them to follow randomly generated trajectories. Our key insight is to leverage a humanoid motion representation that provides human-like motor skills and significantly speeds up training. Using only simplistic reward, state, and object representations, our method shows favorable scalability on diverse objects and trajectories. For training, we do not need a dataset of paired full-body motion and object trajectories. At test time, we only require the object mesh and desired trajectories for grasping and transporting. To demonstrate the capabilities of our method, we show state-of-the-art success rates in following object trajectories and generalizing to unseen objects. Code and models will be released. △ Less

Submitted 18 May, 2025; v1 submitted 16 July, 2024; originally announced July 2024.

Comments: Project page: https://www.zhengyiluo.com/Omnigrasp/

Real-Time Simulated Avatar from Head-Mounted Sensors

Authors: Zhengyi Luo, Jinkun Cao, Rawal Khirodkar, Alexander Winkler, Jing Huang, Kris Kitani, Weipeng Xu

Abstract: We present SimXR, a method for controlling a simulated avatar from information (headset pose and cameras) obtained from AR / VR headsets. Due to the challenging viewpoint of head-mounted cameras, the human body is often clipped out of view, making traditional image-based egocentric pose estimation challenging. On the other hand, headset poses provide valuable information about overall body motion,… ▽ More We present SimXR, a method for controlling a simulated avatar from information (headset pose and cameras) obtained from AR / VR headsets. Due to the challenging viewpoint of head-mounted cameras, the human body is often clipped out of view, making traditional image-based egocentric pose estimation challenging. On the other hand, headset poses provide valuable information about overall body motion, but lack fine-grained details about the hands and feet. To synergize headset poses with cameras, we control a humanoid to track headset movement while analyzing input images to decide body movement. When body parts are seen, the movements of hands and feet will be guided by the images; when unseen, the laws of physics guide the controller to generate plausible motion. We design an end-to-end method that does not rely on any intermediate representations and learns to directly map from images and headset poses to humanoid control signals. To train our method, we also propose a large-scale synthetic dataset created using camera configurations compatible with a commercially available VR headset (Quest 2) and show promising results on real-world captures. To demonstrate the applicability of our framework, we also test it on an AR headset with a forward-facing camera. △ Less

Submitted 24 April, 2024; v1 submitted 11 March, 2024; originally announced March 2024.

Comments: CVPR 2024 Hightlight. Website: https://www.zhengyiluo.com/SimXR/

RoHM: Robust Human Motion Reconstruction via Diffusion

Authors: Siwei Zhang, Bharat Lal Bhatnagar, Yuanlu Xu, Alexander Winkler, Petr Kadlecek, Siyu Tang, Federica Bogo

Abstract: We propose RoHM, an approach for robust 3D human motion reconstruction from monocular RGB(-D) videos in the presence of noise and occlusions. Most previous approaches either train neural networks to directly regress motion in 3D or learn data-driven motion priors and combine them with optimization at test time. The former do not recover globally coherent motion and fail under occlusions; the latte… ▽ More We propose RoHM, an approach for robust 3D human motion reconstruction from monocular RGB(-D) videos in the presence of noise and occlusions. Most previous approaches either train neural networks to directly regress motion in 3D or learn data-driven motion priors and combine them with optimization at test time. The former do not recover globally coherent motion and fail under occlusions; the latter are time-consuming, prone to local minima, and require manual tuning. To overcome these shortcomings, we exploit the iterative, denoising nature of diffusion models. RoHM is a novel diffusion-based motion model that, conditioned on noisy and occluded input data, reconstructs complete, plausible motions in consistent global coordinates. Given the complexity of the problem -- requiring one to address different tasks (denoising and infilling) in different solution spaces (local and global motion) -- we decompose it into two sub-tasks and learn two models, one for global trajectory and one for local motion. To capture the correlations between the two, we then introduce a novel conditioning module, combining it with an iterative inference scheme. We apply RoHM to a variety of tasks -- from motion reconstruction and denoising to spatial and temporal infilling. Extensive experiments on three popular datasets show that our method outperforms state-of-the-art approaches qualitatively and quantitatively, while being faster at test time. The code is available at https://sanweiliti.github.io/ROHM/ROHM.html. △ Less

Submitted 15 April, 2024; v1 submitted 16 January, 2024; originally announced January 2024.

Comments: With the appendix included

Comparing Data-Driven and Mechanistic Models for Predicting Phenology in Deciduous Broadleaf Forests

Authors: Christian Reimers, David Hafezi Rachti, Guahua Liu, Alexander J. Winkler

Abstract: Understanding the future climate is crucial for informed policy decisions on climate change prevention and mitigation. Earth system models play an important role in predicting future climate, requiring accurate representation of complex sub-processes that span multiple time scales and spatial scales. One such process that links seasonal and interannual climate variability to cyclical biological ev… ▽ More Understanding the future climate is crucial for informed policy decisions on climate change prevention and mitigation. Earth system models play an important role in predicting future climate, requiring accurate representation of complex sub-processes that span multiple time scales and spatial scales. One such process that links seasonal and interannual climate variability to cyclical biological events is tree phenology in deciduous broadleaf forests. Phenological dates, such as the start and end of the growing season, are critical for understanding the exchange of carbon and water between the biosphere and the atmosphere. Mechanistic prediction of these dates is challenging. Hybrid modelling, which integrates data-driven approaches into complex models, offers a solution. In this work, as a first step towards this goal, train a deep neural network to predict a phenological index from meteorological time series. We find that this approach outperforms traditional process-based models. This highlights the potential of data-driven methods to improve climate predictions. We also analyze which variables and aspects of the time series influence the predicted onset of the season, in order to gain a better understanding of the advantages and limitations of our model. △ Less

Submitted 8 January, 2024; originally announced January 2024.

Introducing a Deep Neural Network-based Model Predictive Control Framework for Rapid Controller Implementation

Authors: David C. Gordon, Alexander Winkler, Julian Bedei, Patrick Schaber, Jakob Andert, Charles R. Koch

Abstract: Model Predictive Control (MPC) provides an optimal control solution based on a cost function while allowing for the implementation of process constraints. As a model-based optimal control technique, the performance of MPC strongly depends on the model used where a trade-off between model computation time and prediction performance exists. One solution is the integration of MPC with a machine learn… ▽ More Model Predictive Control (MPC) provides an optimal control solution based on a cost function while allowing for the implementation of process constraints. As a model-based optimal control technique, the performance of MPC strongly depends on the model used where a trade-off between model computation time and prediction performance exists. One solution is the integration of MPC with a machine learning (ML) based process model which are quick to evaluate online. This work presents the experimental implementation of a deep neural network (DNN) based nonlinear MPC for Homogeneous Charge Compression Ignition (HCCI) combustion control. The DNN model consists of a Long Short-Term Memory (LSTM) network surrounded by fully connected layers which was trained using experimental engine data and showed acceptable prediction performance with under 5% error for all outputs. Using this model, the MPC is designed to track the Indicated Mean Effective Pressure (IMEP) and combustion phasing trajectories, while minimizing several parameters. Using the acados software package to enable the real-time implementation of the MPC on an ARM Cortex A72, the optimization calculations are completed within 1.4 ms. The external A72 processor is integrated with the prototyping engine controller using a UDP connection allowing for rapid experimental deployment of the NMPC. The IMEP trajectory following of the developed controller was excellent, with a root-mean-square error of 0.133 bar, in addition to observing process constraints. △ Less

Submitted 12 October, 2023; originally announced October 2023.

Comments: Submitted to 2024 American Control Conference (ACC), July 8-12, 2024 in Toronto, Canada. ACC is the annual conference of the American Automatic Control Council (AACC), the U.S. national member organization of the International Federation for Automatic Control (IFAC)

Universal Humanoid Motion Representations for Physics-Based Control

Authors: Zhengyi Luo, Jinkun Cao, Josh Merel, Alexander Winkler, Jing Huang, Kris Kitani, Weipeng Xu

Abstract: We present a universal motion representation that encompasses a comprehensive range of motor skills for physics-based humanoid control. Due to the high dimensionality of humanoids and the inherent difficulties in reinforcement learning, prior methods have focused on learning skill embeddings for a narrow range of movement styles (e.g. locomotion, game characters) from specialized motion datasets.… ▽ More We present a universal motion representation that encompasses a comprehensive range of motor skills for physics-based humanoid control. Due to the high dimensionality of humanoids and the inherent difficulties in reinforcement learning, prior methods have focused on learning skill embeddings for a narrow range of movement styles (e.g. locomotion, game characters) from specialized motion datasets. This limited scope hampers their applicability in complex tasks. We close this gap by significantly increasing the coverage of our motion representation space. To achieve this, we first learn a motion imitator that can imitate all of human motion from a large, unstructured motion dataset. We then create our motion representation by distilling skills directly from the imitator. This is achieved by using an encoder-decoder structure with a variational information bottleneck. Additionally, we jointly learn a prior conditioned on proprioception (humanoid's own pose and velocities) to improve model expressiveness and sampling efficiency for downstream tasks. By sampling from the prior, we can generate long, stable, and diverse human motions. Using this latent space for hierarchical RL, we show that our policies solve tasks using human-like behavior. We demonstrate the effectiveness of our motion representation by solving generative tasks (e.g. strike, terrain traversal) and motion tracking using VR controllers. △ Less

Submitted 11 April, 2024; v1 submitted 6 October, 2023; originally announced October 2023.

Comments: ICLR 2024 Spotlight. Project page: https://zhengyiluo.github.io/PULSE/

Physics-based Motion Retargeting from Sparse Inputs

Authors: Daniele Reda, Jungdam Won, Yuting Ye, Michiel van de Panne, Alexander Winkler

Abstract: Avatars are important to create interactive and immersive experiences in virtual worlds. One challenge in animating these characters to mimic a user's motion is that commercial AR/VR products consist only of a headset and controllers, providing very limited sensor data of the user's pose. Another challenge is that an avatar might have a different skeleton structure than a human and the mapping bet… ▽ More Avatars are important to create interactive and immersive experiences in virtual worlds. One challenge in animating these characters to mimic a user's motion is that commercial AR/VR products consist only of a headset and controllers, providing very limited sensor data of the user's pose. Another challenge is that an avatar might have a different skeleton structure than a human and the mapping between them is unclear. In this work we address both of these challenges. We introduce a method to retarget motions in real-time from sparse human sensor data to characters of various morphologies. Our method uses reinforcement learning to train a policy to control characters in a physics simulator. We only require human motion capture data for training, without relying on artist-generated animations for each avatar. This allows us to use large motion capture datasets to train general policies that can track unseen users from real and sparse data in real-time. We demonstrate the feasibility of our approach on three characters with different skeleton structure: a dinosaur, a mouse-like creature and a human. We show that the avatar poses often match the user surprisingly well, despite having no sensor information of the lower body available. We discuss and ablate the important components in our framework, specifically the kinematic retargeting step, the imitation, contact and action reward as well as our asymmetric actor-critic observations. We further explore the robustness of our method in a variety of settings including unbalancing, dancing and sports motions. △ Less

Submitted 4 July, 2023; originally announced July 2023.

Comments: More info at: https://www.cs.ubc.ca/~dreda/retargeting.html

QuestEnvSim: Environment-Aware Simulated Motion Tracking from Sparse Sensors

Authors: Sunmin Lee, Sebastian Starke, Yuting Ye, Jungdam Won, Alexander Winkler

Abstract: Replicating a user's pose from only wearable sensors is important for many AR/VR applications. Most existing methods for motion tracking avoid environment interaction apart from foot-floor contact due to their complex dynamics and hard constraints. However, in daily life people regularly interact with their environment, e.g. by sitting on a couch or leaning on a desk. Using Reinforcement Learning,… ▽ More Replicating a user's pose from only wearable sensors is important for many AR/VR applications. Most existing methods for motion tracking avoid environment interaction apart from foot-floor contact due to their complex dynamics and hard constraints. However, in daily life people regularly interact with their environment, e.g. by sitting on a couch or leaning on a desk. Using Reinforcement Learning, we show that headset and controller pose, if combined with physics simulation and environment observations can generate realistic full-body poses even in highly constrained environments. The physics simulation automatically enforces the various constraints necessary for realistic poses, instead of manually specifying them as in many kinematic approaches. These hard constraints allow us to achieve high-quality interaction motions without typical artifacts such as penetration or contact sliding. We discuss three features, the environment representation, the contact reward and scene randomization, crucial to the performance of the method. We demonstrate the generality of the approach through various examples, such as sitting on chairs, a couch and boxes, stepping over boxes, rocking a chair and turning an office chair. We believe these are some of the highest-quality results achieved for motion tracking from sparse sensor with scene interaction. △ Less

Submitted 9 June, 2023; originally announced June 2023.

ACM Class: I.3.6

Journal ref: SIGGRAPH 23 Conference Proceedings, August 6-10, 2023, Los Angeles, CA, USA

Perpetual Humanoid Control for Real-time Simulated Avatars

Authors: Zhengyi Luo, Jinkun Cao, Alexander Winkler, Kris Kitani, Weipeng Xu

Abstract: We present a physics-based humanoid controller that achieves high-fidelity motion imitation and fault-tolerant behavior in the presence of noisy input (e.g. pose estimates from video or generated from language) and unexpected falls. Our controller scales up to learning ten thousand motion clips without using any external stabilizing forces and learns to naturally recover from fail-state. Given ref… ▽ More We present a physics-based humanoid controller that achieves high-fidelity motion imitation and fault-tolerant behavior in the presence of noisy input (e.g. pose estimates from video or generated from language) and unexpected falls. Our controller scales up to learning ten thousand motion clips without using any external stabilizing forces and learns to naturally recover from fail-state. Given reference motion, our controller can perpetually control simulated avatars without requiring resets. At its core, we propose the progressive multiplicative control policy (PMCP), which dynamically allocates new network capacity to learn harder and harder motion sequences. PMCP allows efficient scaling for learning from large-scale motion databases and adding new tasks, such as fail-state recovery, without catastrophic forgetting. We demonstrate the effectiveness of our controller by using it to imitate noisy poses from video-based pose estimators and language-based motion generators in a live and real-time multi-person avatar use case. △ Less

Submitted 11 September, 2023; v1 submitted 10 May, 2023; originally announced May 2023.

Comments: ICCV 2023. Project page: https://zhengyiluo.github.io/PHC/

Complementary Textures. A Novel Approach to Object Alignment in Mixed Reality

Authors: Alejandro Martin-Gomez, Alexander Winkler, Rafael de la Tijera Obert, Javad Fotouhi, Daniel Roth, Ulrich Eck, Nassir Navab

Abstract: Alignment between real and virtual objects is a challenging task required for the deployment of Mixed Reality (MR) into manufacturing, medical, and construction applications. To face this challenge, a series of methods have been proposed. While many approaches use dynamic augmentations such as animations, arrows, or text to assist users, they require tracking the position of real objects. In contr… ▽ More Alignment between real and virtual objects is a challenging task required for the deployment of Mixed Reality (MR) into manufacturing, medical, and construction applications. To face this challenge, a series of methods have been proposed. While many approaches use dynamic augmentations such as animations, arrows, or text to assist users, they require tracking the position of real objects. In contrast, when tracking of the real objects is not available or desired, alternative approaches use virtual replicas of real objects to allow for interactive, perceptual virtual-to-real, and/or real-to-virtual alignment. In these cases, the accuracy achieved strongly depends on the quality of the perceptual information provided to the user. This paper proposes a novel set of perceptual alignment concepts that go beyond the use of traditional visualization of virtual replicas, introducing the concept of COMPLEMENTARY TEXTURES to improve interactive alignment in MR applications. To showcase the advantages of using COMPLEMENTARY TEXTURES, we describe three different implementations that provide highly salient visual cues when misalignment is observed; or present semantic augmentations that, when combined with a real object, provide contextual information that can be used during the alignment process. The authors aim to open new paths for the community to explore rather than describing end-to-end solutions. The objective is to show the multitude of opportunities such concepts could provide for further research and development. △ Less

Submitted 16 November, 2022; originally announced November 2022.

QuestSim: Human Motion Tracking from Sparse Sensors with Simulated Avatars

Authors: Alexander Winkler, Jungdam Won, Yuting Ye

Abstract: Real-time tracking of human body motion is crucial for interactive and immersive experiences in AR/VR. However, very limited sensor data about the body is available from standalone wearable devices such as HMDs (Head Mounted Devices) or AR glasses. In this work, we present a reinforcement learning framework that takes in sparse signals from an HMD and two controllers, and simulates plausible and p… ▽ More Real-time tracking of human body motion is crucial for interactive and immersive experiences in AR/VR. However, very limited sensor data about the body is available from standalone wearable devices such as HMDs (Head Mounted Devices) or AR glasses. In this work, we present a reinforcement learning framework that takes in sparse signals from an HMD and two controllers, and simulates plausible and physically valid full body motions. Using high quality full body motion as dense supervision during training, a simple policy network can learn to output appropriate torques for the character to balance, walk, and jog, while closely following the input signals. Our results demonstrate surprisingly similar leg motions to ground truth without any observations of the lower body, even when the input is only the 6D transformations of the HMD. We also show that a single policy can be robust to diverse locomotion styles, different body sizes, and novel environments. △ Less

Submitted 19 September, 2022; originally announced September 2022.

Journal ref: SIGGRAPH Asia 2022 Conference Papers, December 6 to 9, 2022, Daegu, Republic of Korea

EFI: A Toolbox for Feature Importance Fusion and Interpretation in Python

Authors: Aayush Kumar, Jimiama Mafeni Mase, Divish Rengasamy, Benjamin Rothwell, Mercedes Torres Torres, David A. Winkler, Grazziela P. Figueredo

Abstract: This paper presents an open-source Python toolbox called Ensemble Feature Importance (EFI) to provide machine learning (ML) researchers, domain experts, and decision makers with robust and accurate feature importance quantification and more reliable mechanistic interpretation of feature importance for prediction problems using fuzzy sets. The toolkit was developed to address uncertainties in featu… ▽ More This paper presents an open-source Python toolbox called Ensemble Feature Importance (EFI) to provide machine learning (ML) researchers, domain experts, and decision makers with robust and accurate feature importance quantification and more reliable mechanistic interpretation of feature importance for prediction problems using fuzzy sets. The toolkit was developed to address uncertainties in feature importance quantification and lack of trustworthy feature importance interpretation due to the diverse availability of machine learning algorithms, feature importance calculation methods, and dataset dependencies. EFI merges results from multiple machine learning models with different feature importance calculation approaches using data bootstrapping and decision fusion techniques, such as mean, majority voting and fuzzy logic. The main attributes of the EFI toolbox are: (i) automatic optimisation of ML algorithms, (ii) automatic computation of a set of feature importance coefficients from optimised ML algorithms and feature importance calculation techniques, (iii) automatic aggregation of importance coefficients using multiple decision fusion techniques, and (iv) fuzzy membership functions that show the importance of each feature to the prediction task. The key modules and functions of the toolbox are described, and a simple example of their application is presented using the popular Iris dataset. △ Less

Submitted 8 August, 2022; originally announced August 2022.

Comments: 16 pages, 5 tables, 9 figures

Transformer Inertial Poser: Real-time Human Motion Reconstruction from Sparse IMUs with Simultaneous Terrain Generation

Authors: Yifeng Jiang, Yuting Ye, Deepak Gopinath, Jungdam Won, Alexander W. Winkler, C. Karen Liu

Abstract: Real-time human motion reconstruction from a sparse set of (e.g. six) wearable IMUs provides a non-intrusive and economic approach to motion capture. Without the ability to acquire position information directly from IMUs, recent works took data-driven approaches that utilize large human motion datasets to tackle this under-determined problem. Still, challenges remain such as temporal consistency,… ▽ More Real-time human motion reconstruction from a sparse set of (e.g. six) wearable IMUs provides a non-intrusive and economic approach to motion capture. Without the ability to acquire position information directly from IMUs, recent works took data-driven approaches that utilize large human motion datasets to tackle this under-determined problem. Still, challenges remain such as temporal consistency, drifting of global and joint motions, and diverse coverage of motion types on various terrains. We propose a novel method to simultaneously estimate full-body motion and generate plausible visited terrain from only six IMU sensors in real-time. Our method incorporates 1. a conditional Transformer decoder model giving consistent predictions by explicitly reasoning prediction history, 2. a simple yet general learning target named "stationary body points" (SBPs) which can be stably predicted by the Transformer model and utilized by analytical routines to correct joint and global drifting, and 3. an algorithm to generate regularized terrain height maps from noisy SBP predictions which can in turn correct noisy global motion estimation. We evaluate our framework extensively on synthesized and real IMU data, and with real-time live demos, and show superior performance over strong baseline methods. △ Less

Submitted 8 December, 2022; v1 submitted 29 March, 2022; originally announced March 2022.

Comments: SIGGRAPH Asia 2022. Video: https://youtu.be/rXb6SaXsnc0. Code: https://github.com/jyf588/transformer-inertial-poser

Mechanistic Interpretation of Machine Learning Inference: A Fuzzy Feature Importance Fusion Approach

Authors: Divish Rengasamy, Jimiama M. Mase, Mercedes Torres Torres, Benjamin Rothwell, David A. Winkler, Grazziela P. Figueredo

Abstract: With the widespread use of machine learning to support decision-making, it is increasingly important to verify and understand the reasons why a particular output is produced. Although post-training feature importance approaches assist this interpretation, there is an overall lack of consensus regarding how feature importance should be quantified, making explanations of model predictions unreliable… ▽ More With the widespread use of machine learning to support decision-making, it is increasingly important to verify and understand the reasons why a particular output is produced. Although post-training feature importance approaches assist this interpretation, there is an overall lack of consensus regarding how feature importance should be quantified, making explanations of model predictions unreliable. In addition, many of these explanations depend on the specific machine learning approach employed and on the subset of data used when calculating feature importance. A possible solution to improve the reliability of explanations is to combine results from multiple feature importance quantifiers from different machine learning approaches coupled with re-sampling. Current state-of-the-art ensemble feature importance fusion uses crisp techniques to fuse results from different approaches. There is, however, significant loss of information as these approaches are not context-aware and reduce several quantifiers to a single crisp output. More importantly, their representation of 'importance' as coefficients is misleading and incomprehensible to end-users and decision makers. Here we show how the use of fuzzy data fusion methods can overcome some of the important limitations of crisp fusion methods. △ Less

Submitted 22 October, 2021; originally announced October 2021.

Comments: 12 pages, 11 figures, 8 tables

Pose to Seat: Automated Design of Body-Supporting Surfaces

Authors: Kurt Leimer, Andreas Winkler, Stefan Ohrhallinger, Przemyslaw Musialski

Abstract: The design of functional seating furniture is a complicated process which often requires extensive manual design effort and empirical evaluation. We propose a computational design framework for pose-driven automated generation of body-supports which are optimized for comfort of sitting. Given a human body in a specified pose as input, our method computes an approximate pressure distribution that a… ▽ More The design of functional seating furniture is a complicated process which often requires extensive manual design effort and empirical evaluation. We propose a computational design framework for pose-driven automated generation of body-supports which are optimized for comfort of sitting. Given a human body in a specified pose as input, our method computes an approximate pressure distribution that also takes frictional forces and body torques into consideration which serves as an objective measure of comfort. Utilizing this information to find out where the body needs to be supported in order to maintain comfort of sitting, our algorithm can create a supporting mesh suited for a person in that specific pose. This is done in an automated fitting process, using a template model capable of supporting a large variety of sitting poses. The results can be used directly or can be considered as a starting point for further interactive design. △ Less

Submitted 22 March, 2020; originally announced March 2020.

Comments: 14 pages, 15 figures

Journal ref: Computer Aided Geometric Design Volume 79, May 2020, 101855

Impressive computational acceleration by using machine learning for 2-dimensional super-lubricant materials discovery

Authors: Marco Fronzi, Mutaz Abu Ghazaleh, Olexandr Isayev, David A. Winkler, Joe Shapter, Michael J. Ford

Abstract: The screening of novel materials is an important topic in the field of materials science. Although traditional computational modeling, especially first-principles approaches, is a very useful and accurate tool to predict the properties of novel materials, it still demands extensive and expensive state-of-the-art computational resources. Additionally, they can be often extremely time consuming. We… ▽ More The screening of novel materials is an important topic in the field of materials science. Although traditional computational modeling, especially first-principles approaches, is a very useful and accurate tool to predict the properties of novel materials, it still demands extensive and expensive state-of-the-art computational resources. Additionally, they can be often extremely time consuming. We describe a time and resource-efficient machine learning approach to create a large dataset of structural properties of van der Waals layered structures. In particular, we focus on the interlayer energy and the elastic constant of layered materials composed of two different 2-dimensional (2D) structures, that are important for novel solid lubricant and super-lubricant materials. We show that machine learning models can recapitulate results of computationally expansive approaches (i.e. density functional theory) with high accuracy. △ Less

Submitted 29 July, 2020; v1 submitted 20 November, 2019; originally announced November 2019.

Planning and Execution of Dynamic Whole-Body Locomotion for a Hydraulic Quadruped on Challenging Terrain

Authors: Alexander W. Winkler, Carlos Mastalli, Ioannis Havoutis, Michele Focchi, Darwin G. Caldwell, Claudio Semini

Abstract: We present a framework for dynamic quadrupedal locomotion over challenging terrain, where the choice of appropriate footholds is crucial for the success of the behaviour. We build a model of the environment on-line and on-board using an efficient occupancy grid representation. We use Any-time-Repairing A* (ARA*) to search over a tree of possible actions, choose a rough body path and select the loc… ▽ More We present a framework for dynamic quadrupedal locomotion over challenging terrain, where the choice of appropriate footholds is crucial for the success of the behaviour. We build a model of the environment on-line and on-board using an efficient occupancy grid representation. We use Any-time-Repairing A* (ARA*) to search over a tree of possible actions, choose a rough body path and select the locally-best footholds accordingly. We run a n-step lookahead optimization of the body trajectory using a dynamic stability metric, the Zero Moment Point (ZMP), that generates natural dynamic whole-body motions. A combination of floating-base inverse dynamics and virtual model control accurately executes the desired motions on an actively compliant system. Experimental trials show that this framework allows us to traverse terrains at nearly 6 times the speed of our previous work, evaluated over the same set of trials. △ Less

Submitted 7 April, 2019; originally announced April 2019.

Comments: 7 pages, IEEE International Conference on Robotics and Automation

Journal ref: 2015

On-line and on-board planning and perception for quadrupedal locomotion

Authors: Carlos Mastalli, Ioannis Havoutis, Alexander W. Winkler, Darwin G. Caldwell, Claudio Semini

Abstract: We present a legged motion planning approach for quadrupedal locomotion over challenging terrain. We decompose the problem into body action planning and footstep planning. We use a lattice representation together with a set of defined body movement primitives for computing a body action plan. The lattice representation allows us to plan versatile movements that ensure feasibility for every possibl… ▽ More We present a legged motion planning approach for quadrupedal locomotion over challenging terrain. We decompose the problem into body action planning and footstep planning. We use a lattice representation together with a set of defined body movement primitives for computing a body action plan. The lattice representation allows us to plan versatile movements that ensure feasibility for every possible plan. To this end, we propose a set of rules that define the footstep search regions and footstep sequence given a body action. We use Anytime Repairing A* (ARA*) search that guarantees bounded suboptimal plans. Our main contribution is a planning approach that generates on-line versatile movements. Experimental trials demonstrate the performance of our planning approach in a set of challenging terrain conditions. The terrain information and plans are computed on-line and on-board. △ Less

Submitted 7 April, 2019; originally announced April 2019.

Comments: 7 pages, International Conference on Technologies for Practical Robot Applications

Journal ref: published 2015

On-the-fly Augmented Reality for Orthopaedic Surgery Using a Multi-Modal Fiducial

Authors: Sebastian Andress, Alex Johnson, Mathias Unberath, Alexander Winkler, Kevin Yu, Javad Fotouhi, Simon Weidert, Greg Osgood, Nassir Navab

Abstract: Fluoroscopic X-ray guidance is a cornerstone for percutaneous orthopaedic surgical procedures. However, two-dimensional observations of the three-dimensional anatomy suffer from the effects of projective simplification. Consequently, many X-ray images from various orientations need to be acquired for the surgeon to accurately assess the spatial relations between the patient's anatomy and the surgi… ▽ More Fluoroscopic X-ray guidance is a cornerstone for percutaneous orthopaedic surgical procedures. However, two-dimensional observations of the three-dimensional anatomy suffer from the effects of projective simplification. Consequently, many X-ray images from various orientations need to be acquired for the surgeon to accurately assess the spatial relations between the patient's anatomy and the surgical tools. In this paper, we present an on-the-fly surgical support system that provides guidance using augmented reality and can be used in quasi-unprepared operating rooms. The proposed system builds upon a multi-modality marker and simultaneous localization and mapping technique to co-calibrate an optical see-through head mounted display to a C-arm fluoroscopy system. Then, annotations on the 2D X-ray images can be rendered as virtual objects in 3D providing surgical guidance. We quantitatively evaluate the components of the proposed system, and finally, design a feasibility study on a semi-anthropomorphic phantom. The accuracy of our system was comparable to the traditional image-guided technique while substantially reducing the number of acquired X-ray images as well as procedure time. Our promising results encourage further research on the interaction between virtual and real objects, that we believe will directly benefit the proposed method. Further, we would like to explore the capabilities of our on-the-fly augmented reality support system in a larger study directed towards common orthopaedic interventions. △ Less

Submitted 4 January, 2018; originally announced January 2018.

Comments: S. Andress, A. Johnson, M. Unberath, and A. Winkler have contributed equally and are listed in alphabetical order

Journal ref: J. Med. Imag. 5(2), 2018

Optimal and Learning Control for Autonomous Robots

Authors: Jonas Buchli, Farbod Farshidian, Alexander Winkler, Timothy Sandy, Markus Giftthaler

Abstract: Optimal and Learning Control for Autonomous Robots has been taught in the Robotics, Systems and Controls Masters at ETH Zurich with the aim to teach optimal control and reinforcement learning for closed loop control problems from a unified point of view. The starting point is the formulation of of an optimal control problem and deriving the different types of solutions and algorithms from there. T… ▽ More Optimal and Learning Control for Autonomous Robots has been taught in the Robotics, Systems and Controls Masters at ETH Zurich with the aim to teach optimal control and reinforcement learning for closed loop control problems from a unified point of view. The starting point is the formulation of of an optimal control problem and deriving the different types of solutions and algorithms from there. These lecture notes aim at supporting this unified view with a unified notation wherever possible, and a bit of a translation help to compare the terminology and notation in the different fields. The course assumes basic knowledge of Control Theory, Linear Algebra and Stochastic Calculus. △ Less

Submitted 30 August, 2017; originally announced August 2017.

Comments: Lecture Notes, 101 pages

Fast Trajectory Optimization for Legged Robots using Vertex-based ZMP Constraints

Authors: Alexander W Winkler, Farbod Farshidian, Diego Pardo, Michael Neunert, Jonas Buchli

Abstract: This paper combines the fast Zero-Moment-Point (ZMP) approaches that work well in practice with the broader range of capabilities of a Trajectory Optimization formulation, by optimizing over body motion, footholds and Center of Pressure simultaneously. We introduce a vertex-based representation of the support-area constraint, which can treat arbitrarily oriented point-, line-, and area-contacts un… ▽ More This paper combines the fast Zero-Moment-Point (ZMP) approaches that work well in practice with the broader range of capabilities of a Trajectory Optimization formulation, by optimizing over body motion, footholds and Center of Pressure simultaneously. We introduce a vertex-based representation of the support-area constraint, which can treat arbitrarily oriented point-, line-, and area-contacts uniformly. This generalization allows us to create motions such quadrupedal walking, trotting, bounding, pacing, combinations and transitions between these, limping, bipedal walking and push-recovery all with the same approach. This formulation constitutes a minimal representation of the physical laws (unilateral contact forces) and kinematic restrictions (range of motion) in legged locomotion, which allows us to generate various motion in less than a second. We demonstrate the feasibility of the generated motions on a real quadruped robot. △ Less

Submitted 27 May, 2017; originally announced May 2017.

Comments: currently under review for IEEE RA-L

Robust Whole-Body Motion Control of Legged Robots

Authors: Farbod Farshidian, Edo Jelavić, Alexander W. Winkler, Jonas Buchli

Abstract: We introduce a robust control architecture for the whole-body motion control of torque controlled robots with arms and legs. The method is based on the robust control of contact forces in order to track a planned Center of Mass trajectory. Its appeal lies in the ability to guarantee robust stability and performance despite rigid body model mismatch, actuator dynamics, delays, contact surface stiff… ▽ More We introduce a robust control architecture for the whole-body motion control of torque controlled robots with arms and legs. The method is based on the robust control of contact forces in order to track a planned Center of Mass trajectory. Its appeal lies in the ability to guarantee robust stability and performance despite rigid body model mismatch, actuator dynamics, delays, contact surface stiffness, and unobserved ground profiles. Furthermore, we introduce a task space decomposition approach which removes the coupling effects between contact force controller and the other non-contact controllers. Finally, we verify our control performance on a quadruped robot and compare its performance to a standard inverse dynamics approach on hardware. △ Less

Submitted 7 March, 2017; originally announced March 2017.

Comments: 8 Pages

An Efficient Optimal Planning and Control Framework For Quadrupedal Locomotion

Authors: Farbod Farshidian, Michael Neunert, Alexander W. Winkler, Gonzalo Rey, Jonas Buchli

Abstract: In this paper, we present an efficient Dynamic Programing framework for optimal planning and control of legged robots. First we formulate this problem as an optimal control problem for switched systems. Then we propose a multi--level optimization approach to find the optimal switching times and the optimal continuous control inputs. Through this scheme, the decomposed optimization can potentially… ▽ More In this paper, we present an efficient Dynamic Programing framework for optimal planning and control of legged robots. First we formulate this problem as an optimal control problem for switched systems. Then we propose a multi--level optimization approach to find the optimal switching times and the optimal continuous control inputs. Through this scheme, the decomposed optimization can potentially be done more efficiently than the combined approach. Finally, we present a continuous-time constrained LQR algorithm which simultaneously optimizes the feedforward and feedback controller with $O(n)$ time-complexity. In order to validate our approach, we show the performance of our framework on a quadrupedal robot. We choose the Center of Mass dynamics and the full kinematic formulation as the switched system model where the switching times as well as the contact forces and the joint velocities are optimized for different locomotion tasks such as gap crossing, walking and trotting. △ Less

Submitted 4 March, 2017; v1 submitted 30 September, 2016; originally announced September 2016.

Comments: 8 Pages

Trajectory Optimization Through Contacts and Automatic Gait Discovery for Quadrupeds

Authors: Michael Neunert, Farbod Farshidian, Alexander W. Winkler, Jonas Buchli

Abstract: In this work we present a trajectory Optimization framework for whole-body motion planning through contacts. We demonstrate how the proposed approach can be applied to automatically discover different gaits and dynamic motions on a quadruped robot. In contrast to most previous methods, we do not pre-specify contact switches, timings, points or gait patterns, but they are a direct outcome of the op… ▽ More In this work we present a trajectory Optimization framework for whole-body motion planning through contacts. We demonstrate how the proposed approach can be applied to automatically discover different gaits and dynamic motions on a quadruped robot. In contrast to most previous methods, we do not pre-specify contact switches, timings, points or gait patterns, but they are a direct outcome of the optimization. Furthermore, we optimize over the entire dynamics of the robot, which enables the optimizer to fully leverage the capabilities of the robot. To illustrate the spectrum of achievable motions, here we show eight different tasks, which would require very different control structures when solved with state-of-the-art methods. Using our trajectory Optimization approach, we are solving each task with a simple, high level cost function and without any changes in the control structure. Furthermore, we fully integrated our approach with the robot's control and estimation framework such that optimization can be run online. By demonstrating a rough manipulation task with multiple dynamic contact switches, we exemplarily show how optimized trajectories and control inputs can be directly applied to hardware. △ Less

Submitted 15 July, 2016; originally announced July 2016.

Comments: Video: https://youtu.be/sILuqJBsyKs

Projection based whole body motion planning for legged robots

Authors: Diego Pardo, Michael Neunert, Alexander W. Winkler, Jonas Buchli

Abstract: In this paper we present a new approach for dynamic motion planning for legged robots. We formulate a trajectory optimization problem based on a compact form of the robot dynamics. Such a form is obtained by projecting the rigid body dynamics onto the null space of the Constraint Jacobian. As consequence of the projection, contact forces are removed from the model but their effects are still taken… ▽ More In this paper we present a new approach for dynamic motion planning for legged robots. We formulate a trajectory optimization problem based on a compact form of the robot dynamics. Such a form is obtained by projecting the rigid body dynamics onto the null space of the Constraint Jacobian. As consequence of the projection, contact forces are removed from the model but their effects are still taken into account. This approach permits to solve the optimal control problem of a floating base constrained multibody system while avoiding the use of an explicit contact model. We use direct transcription to numerically solve the optimization. As the contact forces are not part of the decision variables the size of the resultant discrete mathematical program is reduced and therefore solutions can be obtained in a tractable time. Using a predefined sequence of contact configurations (phases), our approach solves motions where contact switches occur. Transitions between phases are automatically resolved without using a model for switching dynamics. We present results on a hydraulic quadruped robot (HyQ), including single phase (standing, crouching) as well as multiple phase (rearing, diagonal leg balancing and stepping) dynamic motions. △ Less

Submitted 6 October, 2015; originally announced October 2015.

Evaluating direct transcription and nonlinear optimization methods for robot motion planning

Authors: Diego Pardo, Lukas Möller, Michael Neunert, Alexander W. Winkler, Jonas Buchli

Abstract: This paper studies existing direct transcription methods for trajectory optimization applied to robot motion planning. There are diverse alternatives for the implementation of direct transcription. In this study we analyze the effects of such alternatives when solving a robotics problem. Different parameters such as integration scheme, number of discretization nodes, initialization strategies and… ▽ More This paper studies existing direct transcription methods for trajectory optimization applied to robot motion planning. There are diverse alternatives for the implementation of direct transcription. In this study we analyze the effects of such alternatives when solving a robotics problem. Different parameters such as integration scheme, number of discretization nodes, initialization strategies and complexity of the problem are evaluated. We measure the performance of the methods in terms of computational time, accuracy and quality of the solution. Additionally, we compare two optimization methodologies frequently used to solve the transcribed problem, namely Sequential Quadratic Programming (SQP) and Interior Point Method (IPM). As a benchmark, we solve different motion tasks on an underactuated and non-minimal-phase ball-balancing robot with a 10 dimensional state space and 3 dimensional input space. Additionally, we validate the results on a simulated 3D quadrotor. Finally, as a verification of using direct transcription methods for trajectory optimization on real robots, we present hardware experiments on a motion task including path constraints and actuation limits. △ Less

Submitted 29 January, 2016; v1 submitted 22 April, 2015; originally announced April 2015.