-
Reactive Stepping for Humanoid Robots using Reinforcement Learning: Application to Standing Push Recovery on the Exoskeleton Atalante
Authors:
Alexis Duburcq,
Fabian Schramm,
Guilhem Boéris,
Nicolas Bredeche,
Yann Chevaleyre
Abstract:
State-of-the-art reinforcement learning is now able to learn versatile locomotion, balancing and push-recovery capabilities for bipedal robots in simulation. Yet, the reality gap has mostly been overlooked and the simulated results hardly transfer to real hardware. Either it is unsuccessful in practice because the physics is over-simplified and hardware limitations are ignored, or regularity is no…
▽ More
State-of-the-art reinforcement learning is now able to learn versatile locomotion, balancing and push-recovery capabilities for bipedal robots in simulation. Yet, the reality gap has mostly been overlooked and the simulated results hardly transfer to real hardware. Either it is unsuccessful in practice because the physics is over-simplified and hardware limitations are ignored, or regularity is not guaranteed, and unexpected hazardous motions can occur. This paper presents a reinforcement learning framework capable of learning robust standing push recovery for bipedal robots that smoothly transfer to reality, providing only instantaneous proprioceptive observations. By combining original termination conditions and policy smoothness conditioning, we achieve stable learning, sim-to-real transfer and safety using a policy without memory nor explicit history. Reward engineering is then used to give insights into how to keep balance. We demonstrate its performance in reality on the lower-limb medical exoskeleton Atalante.
△ Less
Submitted 31 July, 2022; v1 submitted 2 March, 2022;
originally announced March 2022.
-
Online Trajectory Planning Through Combined Trajectory Optimization and Function Approximation: Application to the Exoskeleton Atalante
Authors:
Alexis Duburcq,
Yann Chevaleyre,
Nicolas Bredeche,
Guilhem Boéris
Abstract:
Autonomous robots require online trajectory planning capability to operate in the real world. Efficient offline trajectory planning methods already exist, but are computationally demanding, preventing their use online. In this paper, we present a novel algorithm called Guided Trajectory Learning that learns a function approximation of solutions computed through trajectory optimization while ensuri…
▽ More
Autonomous robots require online trajectory planning capability to operate in the real world. Efficient offline trajectory planning methods already exist, but are computationally demanding, preventing their use online. In this paper, we present a novel algorithm called Guided Trajectory Learning that learns a function approximation of solutions computed through trajectory optimization while ensuring accurate and reliable predictions. This function approximation is then used online to generate trajectories. This algorithm is designed to be easy to implement, and practical since it does not require massive computing power. It is readily applicable to any robotics systems and effortless to set up on real hardware since robust control strategies are usually already available. We demonstrate the computational performance of our algorithm on flat-foot walking with the self-balanced exoskeleton Atalante.
△ Less
Submitted 4 March, 2020; v1 submitted 1 October, 2019;
originally announced October 2019.
-
Stabilization of Exoskeletons through Active Ankle Compensation
Authors:
Thomas Gurriet,
Maegan Tucker,
Claudia Kann,
Guilhem Boeris,
Aaron D. Ames
Abstract:
This paper presents an active stabilization method for a fully actuated lower-limb exoskeleton. The method was tested on the exoskeleton ATALANTE, which was designed and built by the French start-up company Wandercraft. The main objective of this paper is to present a practical method of realizing more robust walking on hardware through active ankle compensation. The nominal gait was generated thr…
▽ More
This paper presents an active stabilization method for a fully actuated lower-limb exoskeleton. The method was tested on the exoskeleton ATALANTE, which was designed and built by the French start-up company Wandercraft. The main objective of this paper is to present a practical method of realizing more robust walking on hardware through active ankle compensation. The nominal gait was generated through the hybrid zero dynamic framework. The ankles are individually controlled to establish three main directives; (1) keeping the non-stance foot parallel to the ground, (2) maintaining rigid contact between the stance foot and the ground, and (3) closing the loop on pelvis orientation to achieve better tracking. Each individual component of this method was demonstrated separately to show each component's contribution to stability. The results showed that the ankle controller was able to experimentally maintain static balance in the sagittal plane while the exoskeleton was balanced on one leg, even when disturbed. The entire ankle controller was then also demonstrated on crutch-less dynamic walking. During testing, an anatomically correct manikin was placed in the exoskeleton, in lieu of a paraplegic patient. The pitch of the pelvis of the exoskeleton-manikin system was shown to track the gait trajectory better when ankle compensation was used. Overall, active ankle compensation was demonstrated experimentally to improve balance in the sagittal plane of the exoskeleton manikin system and points to an improved practical approach for stable walking.
△ Less
Submitted 25 September, 2019;
originally announced September 2019.
-
Towards Variable Assistance for Lower Body Exoskeletons
Authors:
Thomas Gurriet,
Maegan Tucker,
Alexis Duburcq,
Guilhem Boeris,
Aaron D. Ames
Abstract:
This paper presents and experimentally demonstrates a novel framework for variable assistance on lower body exoskeletons, based upon safety-critical control methods. Existing work has shown that providing some freedom of movement around a nominal gait, instead of rigidly following it, accelerates the spinal learning process of people with a walking impediment when using a lower body exoskeleton. W…
▽ More
This paper presents and experimentally demonstrates a novel framework for variable assistance on lower body exoskeletons, based upon safety-critical control methods. Existing work has shown that providing some freedom of movement around a nominal gait, instead of rigidly following it, accelerates the spinal learning process of people with a walking impediment when using a lower body exoskeleton. With this as motivation, we present a method to accurately control how much a subject is allowed to deviate from a given gait while ensuring robustness to patient perturbation. This method leverages control barrier functions to force certain joints to remain inside predefined trajectory tubes in a minimally invasive way. The effectiveness of the method is demonstrated experimentally with able-bodied subjects and the Atalante lower body exoskeleton.
△ Less
Submitted 2 December, 2019; v1 submitted 24 September, 2019;
originally announced September 2019.
-
Feedback Control of an Exoskeleton for Paraplegics: Toward Robustly Stable Hands-free Dynamic Walking
Authors:
Omar Harib,
Ayonga Hereid,
Ayush Agrawal,
Thomas Gurriet,
Sylvain Finet,
Guilhem Boeris,
Alexis Duburcq,
M. Eva Mungai,
Matthieu Masselin,
Aaron D. Ames,
Koushil Sreenath,
Jessy Grizzle
Abstract:
This manuscript presents control of a high-DOF fully actuated lower-limb exoskeleton for paraplegic individuals. The key novelty is the ability for the user to walk without the use of crutches or other external means of stabilization. We harness the power of modern optimization techniques and supervised machine learning to develop a smooth feedback control policy that provides robust velocity regu…
▽ More
This manuscript presents control of a high-DOF fully actuated lower-limb exoskeleton for paraplegic individuals. The key novelty is the ability for the user to walk without the use of crutches or other external means of stabilization. We harness the power of modern optimization techniques and supervised machine learning to develop a smooth feedback control policy that provides robust velocity regulation and perturbation rejection. Preliminary evaluation of the stability and robustness of the proposed approach is demonstrated through the Gazebo simulation environment. In addition, preliminary experimental results with (complete) paraplegic individuals are included for the previous version of the controller.
△ Less
Submitted 21 May, 2018; v1 submitted 22 February, 2018;
originally announced February 2018.
