1: \begin{abstract}

2: Adversarial neural networks solve many important problems in data science, but are notoriously difficult to train.  These difficulties come from the fact that optimal weights for adversarial nets correspond to saddle points, and not minimizers, of the loss function.  The alternating stochastic gradient methods typically used for such problems do not reliably converge to saddle points, and when convergence does happen it is often highly sensitive to learning rates.  We propose a simple modification of stochastic gradient descent that stabilizes adversarial networks.  We show, both in theory and practice, that the proposed method reliably converges to saddle points, and is stable with a wider range of training parameters than a non-prediction method.  This makes adversarial networks less likely to ``collapse,''  and enables faster training with larger learning rates.

3: %Adversarial networks are notoriously difficult to train because their training involves alternating minimization and maximization operations on multi-task objective function. We present a prediction method for stochastic solvers in which the weights updates for one operation is accomplished by looking ahead and predicting the future path of alternate operation. The theoretical analysis shows that our proposed method is asymptotically stable for a class of saddle point problems. A distinguishing aspect of our method is that without any computation overhead, architecture re-engineering and objective re-modeling, is it known to work across existing stochastic gradient solvers. Experiments on wide range of applications involving adversarial networks demonstrates that the presented algorithm outperforms state-of-the-art approaches and provides greater flexibility in the choice of the hyper-parameters. This enables faster training using large learning rates without being plagued by any instability problems. 

4: \end{abstract}

5: