1: \begin{abstract}

2: 

3: This paper proposes a new optimizer for deep learning, named d-AmsGrad.

4: In the real-world data, noise and outliers cannot be excluded from dataset to be used for learning robot skills.

5: This problem is especially striking for robots that learn by collecting data in real time, which cannot be sorted manually.

6: Several noise-robust optimizers have therefore been developed to resolve this problem, and one of them, named AmsGrad, which is a variant of Adam optimizer, has a proof of its convergence.

7: However, in practice, it does not improve learning performance in robotics scenarios.

8: This reason is hypothesized that most of robot learning problems are non-stationary, but AmsGrad assumes the maximum second momentum during learning to be stationarily given.

9: In order to adapt to the non-stationary problems, an improved version, which slowly decays the maximum second momentum, is proposed.

10: The proposed optimizer has the same capability of reaching the global optimum as baselines, and its performance outperformed that of the baselines in robotics problems.

11: 

12: \end{abstract}