85a6c4a693c42304.tex
1: \begin{abstract}
2: %\begin{flushleft}
3: We study the global convergence of generative adversarial imitation learning for linear quadratic regulators, which is posed as minimax optimization. To address the challenges arising from non-convex-concave geometry, we analyze the alternating gradient algorithm and establish its Q-linear rate of convergence to a unique saddle point, which simultaneously recovers the globally optimal policy and reward function. We hope our results may serve as a small step towards understanding and taming the instability in imitation learning as well as in more general non-convex-concave alternating minimax optimization that arises from reinforcement learning and generative adversarial learning. 
4: %\end{flushleft}
5: 
6: %A case study of imitation learning in linear quadratic control setting is presented. We adopt the Generative Adversarial Imitation Learning (GAIL) framework for imitation learning and prove global convergence of the gradient algorithm for Linear Quadratic Regulator (LQR). We further show that with suitable choice of constant stepsize, the gradient algorithm converges to the saddle point in Q-linear rate. 
7: \end{abstract}
8: