1: \begin{abstract}

2: We propose a new technique that boosts the convergence of training generative adversarial networks.

3: Generally, the rate of training deep models reduces severely after multiple iterations.

4: A key reason for this phenomenon is that a deep network is expressed using a 

5: highly non-convex finite-dimensional model, and thus

6: the parameter gets stuck in a local optimum.

7: Because of this, methods often suffer not only from degeneration of the convergence speed but also from limitations in the representational power of the trained network.

8: To overcome this issue, we propose an additional layer called the {\it gradient layer}

9: to seek a descent direction in an {\it infinite-dimensional space}.  

10: Because the layer is constructed in the infinite-dimensional space, 

11: we are not restricted by the specific model structure of finite-dimensional models. 

12: As a result, we can get out of the local optima in finite-dimensional models and move towards the global optimal function more directly.

13: In this paper, this phenomenon is explained from the functional gradient method perspective of the gradient layer.

14: Interestingly, the optimization procedure using the gradient layer naturally constructs the deep structure of the network.

15: Moreover, we demonstrate that this procedure can be regarded as a discretization method of the gradient flow that naturally reduces the objective function.

16: Finally, the method is tested using several numerical experiments, which show its fast convergence. 

17: \end{abstract}

18: