-
Safe Reinforcement Learning Using Robust Control Barrier Functions
Authors:
Yousef Emam,
Gennaro Notomista,
Paul Glotfelter,
Zsolt Kira,
Magnus Egerstedt
Abstract:
Reinforcement Learning (RL) has been shown to be effective in many scenarios. However, it typically requires the exploration of a sufficiently large number of state-action pairs, some of which may be unsafe. Consequently, its application to safety-critical systems remains a challenge. An increasingly common approach to address safety involves the addition of a safety layer that projects the RL act…
▽ More
Reinforcement Learning (RL) has been shown to be effective in many scenarios. However, it typically requires the exploration of a sufficiently large number of state-action pairs, some of which may be unsafe. Consequently, its application to safety-critical systems remains a challenge. An increasingly common approach to address safety involves the addition of a safety layer that projects the RL actions onto a safe set of actions. In turn, a difficulty for such frameworks is how to effectively couple RL with the safety layer to improve the learning performance. In this paper, we frame safety as a differentiable robust-control-barrier-function layer in a model-based RL framework. Moreover, we also propose an approach to modularly learn the underlying reward-driven task, independent of safety constraints. We demonstrate that this approach both ensures safety and effectively guides exploration during training in a range of experiments, including zero-shot transfer when the reward is learned in a modular way.
△ Less
Submitted 22 June, 2022; v1 submitted 11 October, 2021;
originally announced October 2021.
-
Data-Driven Robust Barrier Functions for Safe, Long-Term Operation
Authors:
Yousef Emam,
Paul Glotfelter,
Sean Wilson,
Gennaro Notomista,
Magnus Egerstedt
Abstract:
Applications that require multi-robot systems to operate independently for extended periods of time in unknown or unstructured environments face a broad set of challenges, such as hardware degradation, changing weather patterns, or unfamiliar terrain. To operate effectively under these changing conditions, algorithms developed for long-term autonomy applications require a stronger focus on robustn…
▽ More
Applications that require multi-robot systems to operate independently for extended periods of time in unknown or unstructured environments face a broad set of challenges, such as hardware degradation, changing weather patterns, or unfamiliar terrain. To operate effectively under these changing conditions, algorithms developed for long-term autonomy applications require a stronger focus on robustness. Consequently, this work considers the ability to satisfy the operation-critical constraints of a disturbed system in a modular fashion, which means compatibility with different system objectives and disturbance representations. Toward this end, this paper introduces a controller-synthesis approach to constraint satisfaction for disturbed control-affine dynamical systems by utilizing Control Barrier Functions (CBFs). The aforementioned framework is constructed by modelling the disturbance as a union of convex hulls and leveraging previous work on CBFs for differential inclusions. This method of disturbance modeling grants compatibility with different disturbance-estimation methods. For example, this work demonstrates how a disturbance learned via a Gaussian process may be utilized in the proposed framework. These estimated disturbances are incorporated into the proposed controller-synthesis framework which is then tested on a fleet of robots in different scenarios.
△ Less
Submitted 15 April, 2021;
originally announced April 2021.
-
Data-Driven Adaptive Task Allocation for Heterogeneous Multi-Robot Teams Using Robust Control Barrier Functions
Authors:
Yousef Emam,
Gennaro Notomista,
Paul Glotfelter,
Magnus Egerstedt
Abstract:
Multi-robot task allocation is a ubiquitous problem in robotics due to its applicability in a variety of scenarios. Adaptive task-allocation algorithms account for unknown disturbances and unpredicted phenomena in the environment where robots are deployed to execute tasks. However, this adaptivity typically comes at the cost of requiring precise knowledge of robot models in order to evaluate the a…
▽ More
Multi-robot task allocation is a ubiquitous problem in robotics due to its applicability in a variety of scenarios. Adaptive task-allocation algorithms account for unknown disturbances and unpredicted phenomena in the environment where robots are deployed to execute tasks. However, this adaptivity typically comes at the cost of requiring precise knowledge of robot models in order to evaluate the allocation effectiveness and to adjust the task assignment online. As such, environmental disturbances can significantly degrade the accuracy of the models which in turn negatively affects the quality of the task allocation. In this paper, we leverage Gaussian processes, differential inclusions, and robust control barrier functions to learn environmental disturbances in order to guarantee robust task execution. We show the implementation and the effectiveness of the proposed framework on a real multi-robot system.
△ Less
Submitted 10 November, 2020; v1 submitted 2 November, 2020;
originally announced November 2020.
-
Robust Barrier Functions for a Fully Autonomous, Remotely Accessible Swarm-Robotics Testbed
Authors:
Yousef Emam,
Paul Glotfelter,
Magnus Egerstedt
Abstract:
The Robotarium, a remotely accessible swarm-robotics testbed, has provided free, open access to robotics and controls research for hundreds of users in thousands of experiments. This high level of usage requires autonomy in the system, which mainly corresponds to constraint satisfaction in the context of users' submissions. In other words, in case that the users' inputs to the robots may lead to c…
▽ More
The Robotarium, a remotely accessible swarm-robotics testbed, has provided free, open access to robotics and controls research for hundreds of users in thousands of experiments. This high level of usage requires autonomy in the system, which mainly corresponds to constraint satisfaction in the context of users' submissions. In other words, in case that the users' inputs to the robots may lead to collisions, these inputs must be altered to avoid these collisions automatically. However, these alterations must be minimal so as to preserve the users' objective in the experiment. Toward this end, the system has utilized barrier functions, which admit a minimally invasive controller-synthesis procedure. However, barrier functions are yet to be robustified with respect to unmodeled disturbances (e.g., wheel slip or packet loss) in a manner conducive to real-time synthesis. As such, this paper formulates robust barrier functions for a general class of disturbed control-affine systems that, in turn, is key for the Robotarium to operate fully autonomously (i.e., without human supervision). Experimental results showcase the effectiveness of this robust formulation in a long-term experiment in the Robotarium.
△ Less
Submitted 6 September, 2019;
originally announced September 2019.
-
A Parametric MPC Approach to Balancing the Cost of Abstraction for Differential-Drive Mobile Robots
Authors:
Paul Glotfelter,
Magnus Egerstedt
Abstract:
When designing control strategies for differential-drive mobile robots, one standard tool is the consideration of a point at a fixed distance along a line orthogonal to the wheel axis instead of the full pose of the vehicle. This abstraction supports replacing the non-holonomic, three-state unicycle model with a much simpler two-state single-integrator model (i.e., a velocity-controlled point). Ye…
▽ More
When designing control strategies for differential-drive mobile robots, one standard tool is the consideration of a point at a fixed distance along a line orthogonal to the wheel axis instead of the full pose of the vehicle. This abstraction supports replacing the non-holonomic, three-state unicycle model with a much simpler two-state single-integrator model (i.e., a velocity-controlled point). Yet this transformation comes at a performance cost, through the robot's precision and maneuverability. This work contains derivations for expressions of these precision and maneuverability costs in terms of the transformation's parameters. Furthermore, these costs show that only selecting the parameter once over the course of an application may cause an undue loss of precision. Model Predictive Control (MPC) represents one such method to ameliorate this condition. However, MPC typically realizes a control signal, rather than a parameter, so this work also proposes a Parametric Model Predictive Control (PMPC) method for parameter and sampling horizon optimization. Experimental results are presented that demonstrate the effects of the parameterization on the deployment of algorithms developed for the single-integrator model on actual differential-drive mobile robots.
△ Less
Submitted 20 February, 2018;
originally announced February 2018.
