1: \begin{abstract}

2:     %

3: 

4: We introduce a general framework for competitive gradient-based learning 

5: that encompasses a wide breadth of multi-agent learning algorithms, and analyze the limiting behavior of competitive gradient-based learning algorithms using dynamical systems theory. For both general-sum and potential games, we characterize  a

6: non-negligible subset of the local Nash equilibria that will be avoided if each agent employs a gradient-based learning algorithm. We also shed light on the issue of convergence to non-Nash strategies in general- and zero-sum games, which may have no relevance to the underlying game, and arise solely due

7: to the choice of algorithm. The existence and frequency of such strategies may

8: explain some of the difficulties encountered when using gradient descent in

9: zero-sum games as, e.g., in the training of generative adversarial networks. To reinforce the theoretical

10: contributions, we provide empirical results that highlight the frequency of linear quadratic dynamic games (a benchmark for multi-agent reinforcement learning) that admit global Nash equilibria that are almost surely avoided by policy gradient. %

11: 

12:  \end{abstract}

13: