-
A nonlinear real time capable motion cueing algorithm based on deep reinforcement learning
Authors:
Hendrik Scheidel,
Camilo Gonzalez,
Houshyar Asadi,
Tobias Bellmann,
Andreas Seefried,
Shady Mohamed,
Saeid Nahavandi
Abstract:
In motion simulation, motion cueing algorithms are used for the trajectory planning of the motion simulator platform, where workspace limitations prevent direct reproduction of reference trajectories. Strategies such as motion washout, which return the platform to its center, are crucial in these settings. For serial robotic MSPs with highly nonlinear workspaces, it is essential to maximize the ef…
▽ More
In motion simulation, motion cueing algorithms are used for the trajectory planning of the motion simulator platform, where workspace limitations prevent direct reproduction of reference trajectories. Strategies such as motion washout, which return the platform to its center, are crucial in these settings. For serial robotic MSPs with highly nonlinear workspaces, it is essential to maximize the efficient utilization of the MSPs kinematic and dynamic capabilities. Traditional approaches, including classical washout filtering and linear model predictive control, fail to consider platform-specific, nonlinear properties, while nonlinear model predictive control, though comprehensive, imposes high computational demands that hinder real-time, pilot-in-the-loop application without further simplification. To overcome these limitations, we introduce a novel approach using deep reinforcement learning for motion cueing, demonstrated here for the first time in a 6-degree-of-freedom setting with full consideration of the MSPs kinematic nonlinearities. Previous work by the authors successfully demonstrated the application of DRL to a simplified 2-DOF setup, which did not consider kinematic or dynamic constraints. This approach has been extended to all 6 DOF by incorporating a complete kinematic model of the MSP into the algorithm, a crucial step for enabling its application on a real motion simulator. The training of the DRL-MCA is based on Proximal Policy Optimization in an actor-critic implementation combined with an automated hyperparameter optimization. After detailing the necessary training framework and the algorithm itself, we provide a comprehensive validation, demonstrating that the DRL MCA achieves competitive performance against established algorithms. Moreover, it generates feasible trajectories by respecting all system constraints and meets all real-time requirements with low...
△ Less
Submitted 3 April, 2025; v1 submitted 13 March, 2025;
originally announced March 2025.
-
Microparticle-based Controlled Drug Delivery Systems: From Experiments to Statistical Analysis and Design
Authors:
Sebastian Lotter,
Tom Bellmann,
Sophie Marx,
Mara Wesinger,
Lukas Brand,
Maximilian Schäfer,
Dagmar Fischer,
Robert Schober
Abstract:
Controlled drug delivery (CDD), the controlled release and delivery of therapeutic drugs inside the human body, is a promising approach to increase the efficacy of drug administration and reduce harmful side effects to the body. CDD has been a major research focus in the field of molecular communications (MC) with the goal to aid the design and optimization of CDD systems with communication theore…
▽ More
Controlled drug delivery (CDD), the controlled release and delivery of therapeutic drugs inside the human body, is a promising approach to increase the efficacy of drug administration and reduce harmful side effects to the body. CDD has been a major research focus in the field of molecular communications (MC) with the goal to aid the design and optimization of CDD systems with communication theoretical analysis. However, the existing studies of CDD under the MC framework are purely theoretical, and the potential of MC for the development of practical CDD applications remains yet to be shown. This paper presents a step towards filling this research gap. Specifically, we present a novel MC-based model for a specific CDD system in which drugs are embedded into microparticles and released gradually towards the target site. It is demonstrated that the proposed model is able to faithfully reproduce experimental data. Furthermore, statistical analysis is conducted to explore the impact of the microparticle size on the drug release. The presented results reveal the sensitivity of the drug release to changes in the microparticle size. In this way, the proposed model can be used for the design of future microparticle-based CDD systems.
△ Less
Submitted 10 May, 2023; v1 submitted 5 May, 2023;
originally announced May 2023.
-
A Deep Reinforcement Learning Based Motion Cueing Algorithm for Vehicle Driving Simulation
Authors:
Hendrik Scheidel,
Houshyar Asadi,
Tobias Bellmann,
Andreas Seefried,
Shady Mohamed,
Saeid Nahavandi
Abstract:
Motion cueing algorithms (MCA) are used to control the movement of motion simulation platforms (MSP) to reproduce the motion perception of a real vehicle driver as accurately as possible without exceeding the limits of the workspace of the MSP. Existing approaches either produce non-optimal results due to filtering, linearization, or simplifications, or the computational time required exceeds the…
▽ More
Motion cueing algorithms (MCA) are used to control the movement of motion simulation platforms (MSP) to reproduce the motion perception of a real vehicle driver as accurately as possible without exceeding the limits of the workspace of the MSP. Existing approaches either produce non-optimal results due to filtering, linearization, or simplifications, or the computational time required exceeds the real-time requirements of a closed-loop application. This work presents a new solution to the motion cueing problem, where instead of a human designer specifying the principles of the MCA, an artificial intelligence (AI) learns the optimal motion by trial and error in interaction with the MSP. To achieve this, a well-established deep reinforcement learning (RL) algorithm is applied, where an agent interacts with an environment, allowing him to directly control a simulated MSP to obtain feedback on its performance. The RL algorithm used is proximal policy optimization (PPO), where the value function and the policy corresponding to the control strategy are both learned and mapped in artificial neural networks (ANN). This approach is implemented in Python and the functionality is demonstrated by the practical example of pre-recorded lateral maneuvers. The subsequent validation shows that the RL algorithm is able to learn the control strategy and improve the quality of the immersion compared to an established method. Thereby, the perceived motion signals determined by a model of the vestibular system are more accurately reproduced, and the resources of the MSP are used more economically.
△ Less
Submitted 19 March, 2025; v1 submitted 15 April, 2023;
originally announced April 2023.
