1: \begin{abstract}

2: %We investigate the properties of a family of double adaptive stochastic gradient methods that leverage the complementary ideas from the adaptive moment methods, widely used in deep learning, and advancements in adaptive probabilistic methods used for stochastic optimization.

3: Adaptive moment methods have been remarkably successful in deep learning optimization, particularly in the presence of noisy and/or sparse gradients. 

4: We further the advantages of adaptive moment techniques by proposing a family of double adaptive stochastic gradient methods~\textsc{DASGrad}. 

5: They leverage the complementary ideas of the adaptive moment algorithms widely used by deep learning community, and recent advances in adaptive probabilistic algorithms.

6: We analyze the theoretical convergence improvements of our approach in a stochastic convex optimization setting, and provide empirical validation of our findings with convex and non convex objectives. 

7: We observe that the benefits of~\textsc{DASGrad} increase with the model complexity and variability of the gradients, and we explore the resulting utility in extensions of distribution-matching multitask learning. 

8: \end{abstract}

9: