1: \begin{abstract}

2: Minimax optimization plays a key role in adversarial training of machine

3: learning algorithms, such as learning generative models,

4: domain adaptation, privacy preservation, and robust learning.

5: In this paper, we demonstrate the failure of alternating gradient descent

6: in minimax optimization problems due to the discontinuity of solutions of 

7: the inner maximization. 

8: To address this, we propose a new $\epsilon$-subgradient descent algorithm 

9: that addresses this problem by simultaneously tracking $K$ candidate solutions.

10: Practically, the algorithm can find solutions that previous saddle-point

11: algorithms cannot find, with only a sublinear increase of complexity in $K$. 

12: We analyze the conditions under which the algorithm converges to

13: the true solution in detail. 

14: A significant improvement in stability and convergence speed of the algorithm

15: is observed in simple representative problems, GAN training, and domain-adaptation problems.

16: \end{abstract}

17: