-
RobocupGym: A challenging continuous control benchmark in Robocup
Authors:
Michael Beukman,
Branden Ingram,
Geraud Nangue Tasse,
Benjamin Rosman,
Pravesh Ranchod
Abstract:
Reinforcement learning (RL) has progressed substantially over the past decade, with much of this progress being driven by benchmarks. Many benchmarks are focused on video or board games, and a large number of robotics benchmarks lack diversity and real-world applicability. In this paper, we aim to simplify the process of applying reinforcement learning in the 3D simulation league of Robocup, a rob…
▽ More
Reinforcement learning (RL) has progressed substantially over the past decade, with much of this progress being driven by benchmarks. Many benchmarks are focused on video or board games, and a large number of robotics benchmarks lack diversity and real-world applicability. In this paper, we aim to simplify the process of applying reinforcement learning in the 3D simulation league of Robocup, a robotic football competition. To this end, we introduce a Robocup-based RL environment based on the open source rcssserver3d soccer server, simple pre-defined tasks, and integration with a popular RL library, Stable Baselines 3. Our environment enables the creation of high-dimensional continuous control tasks within a robotics football simulation. In each task, an RL agent controls a simulated Nao robot, and can interact with the ball or other agents. We open-source our environment and training code at https://github.com/Michael-Beukman/RobocupGym.
△ Less
Submitted 3 July, 2024;
originally announced July 2024.
-
Learning Options from Demonstration using Skill Segmentation
Authors:
Matthew Cockcroft,
Shahil Mawjee,
Steven James,
Pravesh Ranchod
Abstract:
We present a method for learning options from segmented demonstration trajectories. The trajectories are first segmented into skills using nonparametric Bayesian clustering and a reward function for each segment is then learned using inverse reinforcement learning. From this, a set of inferred trajectories for the demonstration are generated. Option initiation sets and termination conditions are l…
▽ More
We present a method for learning options from segmented demonstration trajectories. The trajectories are first segmented into skills using nonparametric Bayesian clustering and a reward function for each segment is then learned using inverse reinforcement learning. From this, a set of inferred trajectories for the demonstration are generated. Option initiation sets and termination conditions are learned from these trajectories using the one-class support vector machine clustering algorithm. We demonstrate our method in the four rooms domain, where an agent is able to autonomously discover usable options from human demonstration. Our results show that these inferred options can then be used to improve learning and planning.
△ Less
Submitted 19 January, 2020;
originally announced January 2020.
-
Reinforcement Learning with Parameterized Actions
Authors:
Warwick Masson,
Pravesh Ranchod,
George Konidaris
Abstract:
We introduce a model-free algorithm for learning in Markov decision processes with parameterized actions-discrete actions with continuous parameters. At each step the agent must select both which action to use and which parameters to use with that action. We introduce the Q-PAMDP algorithm for learning in these domains, show that it converges to a local optimum, and compare it to direct policy sea…
▽ More
We introduce a model-free algorithm for learning in Markov decision processes with parameterized actions-discrete actions with continuous parameters. At each step the agent must select both which action to use and which parameters to use with that action. We introduce the Q-PAMDP algorithm for learning in these domains, show that it converges to a local optimum, and compare it to direct policy search in the goal-scoring and Platform domains.
△ Less
Submitted 26 November, 2015; v1 submitted 4 September, 2015;
originally announced September 2015.
