Title:Optimal Tracking for Partially-Unknown Continuous Time Nonlinear Systems with Actuator Constraints using Critic-Only Integral Reinforcement Learning

Abstract:Integral reinforcement learning (IRL) was proposed in literature to obviate the requirement of drift dynamics in adaptive dynamic programming (ADP) framework. Most of the online IRL schemes that have been presented in literature require two sets of neural network (NNs), known as actor-critic NN and an initial stabilizing controller. In order to obviate the requirement of initial stabilizing controller and dual-approximator structure, a modified gradient descent-based update law containing a stabilizing term with critic-only structure has recently been proposed for reinforcement learning (RL) algorithm for robust tracking of continuous time nonlinear systems. To the best of the authors' knowledge, there has been no study on leveraging such stabilizing term in IRL algorithm framework to solve optimal trajectory tracking problems for continuous time nonlinear systems with actuator constraints. To this end a novel update law leveraging the stabilizing term along with variable gain gradient descent in IRL framework is presented in this paper. With these modifications, the IRL tracking controller can be implemented using just critic NN without the need of an initial stabilizing controller. The most salient feature of the presented update law is its variable learning rate, which scales the pace of learning based on instantaneous Hamilton-Jacobi-Bellman (HJB) error and rate of variation of Lyapunov function along the system trajectories. The augmented system states and NN weight error are shown to possess uniform ultimate boundedness (UUB) stability under the update law and achieve a tighter residual set. The presented update law is validated on a full 6-DoF nonlinear model of UAV for attitude control.