Showing 1–2 of 2 results for author: McKenzie, D M

Maximum Entropy Hindsight Experience Replay

Authors: Douglas C. Crowder, Matthew L. Trappett, Darrien M. McKenzie, Frances S. Chance

Abstract: Hindsight experience replay (HER) is well-known to accelerate goal-based reinforcement learning (RL). While HER is generally applied to off-policy RL algorithms, we previously showed that HER can also accelerate on-policy algorithms, such as proximal policy optimization (PPO), for goal-based Predator-Prey environments. Here, we show that we can improve the previous PPO-HER algorithm by selectively… ▽ More Hindsight experience replay (HER) is well-known to accelerate goal-based reinforcement learning (RL). While HER is generally applied to off-policy RL algorithms, we previously showed that HER can also accelerate on-policy algorithms, such as proximal policy optimization (PPO), for goal-based Predator-Prey environments. Here, we show that we can improve the previous PPO-HER algorithm by selectively applying HER in a principled manner. △ Less

Submitted 31 October, 2024; originally announced October 2024.

Comments: 11 pages, 11 Figures

Hindsight Experience Replay Accelerates Proximal Policy Optimization

Authors: Douglas C. Crowder, Darrien M. McKenzie, Matthew L. Trappett, Frances S. Chance

Abstract: Hindsight experience replay (HER) accelerates off-policy reinforcement learning algorithms for environments that emit sparse rewards by modifying the goal of the episode post-hoc to be some state achieved during the episode. Because post-hoc modification of the observed goal violates the assumptions of on-policy algorithms, HER is not typically applied to on-policy algorithms. Here, we show that H… ▽ More Hindsight experience replay (HER) accelerates off-policy reinforcement learning algorithms for environments that emit sparse rewards by modifying the goal of the episode post-hoc to be some state achieved during the episode. Because post-hoc modification of the observed goal violates the assumptions of on-policy algorithms, HER is not typically applied to on-policy algorithms. Here, we show that HER can dramatically accelerate proximal policy optimization (PPO), an on-policy reinforcement learning algorithm, when tested on a custom predator-prey environment. △ Less

Submitted 29 October, 2024; originally announced October 2024.

Comments: 12 pages. 10 Figures