1: \begin{abstract}

2: Optimal tracking of continuous time nonlinear systems has been extensively studied in literature. However, in several applications, absence of knowledge about system dynamics poses a severe challenge to solving the optimal tracking problem. 

3: This has found growing attention among researchers recently, and integral reinforcement learning (IRL)-based method augmented with actor neural network (NN) have been deployed to this end. However, very few studies have been directed to model-free $H_{\infty}$ optimal tracking control that helps in attenuating the effect of disturbances on the system performance without any prior knowledge about system dynamics. To this end a recursive least square-based parameter update was recently proposed. However, gradient descent-based parameter update scheme is more sensitive to real-time variation in plant dynamics. And experience replay (ER) technique has been shown to improve the convergence of NN weights by utilizing past observations iteratively. Motivated by these, this paper presents a novel parameter update law based on variable gain gradient descent and experience replay technique for tuning the weights of critic, actor and disturbance NNs. 

4: % The learning rate in variable gain gradient descent is a function of Hamilton-Jacobi-Isaac (HJI) error such that it accelerates the learning process when the HJI error is large and slows down the learning process as the HJI error becomes smaller. 

5: % Combined effect of variable gain gradient descent and experience replay results in faster convergence of NN weights, while these along with three additional robust terms in the update law leads to a significantly tighter residual set, on which error in neural network weights converge to. 

6: The presented update law leads to improved model-free tracking performance under $\mathcal{L}_2$-bounded disturbance. Simulation results are presented to validate the presented update law.

7: \end{abstract}

8: