1: \begin{abstract}

2: In this paper, we propose a new, simplified high probability analysis of AdaGrad for smooth, non-convex problems. 

3: More specifically, we focus on a particular accelerated gradient (AGD) template \citep{lan2020first}, through which we recover the original AdaGrad and its variant with averaging, and prove a convergence rate of $\mathcal O (1/ \sqrt{T})$ with high probability without the knowledge of smoothness and variance. 

4: We use a particular version of Freedman's concentration bound for martingale difference sequences \citep{kakade2008generalization} which enables us to achieve the best-known dependence of $\log (1 / \delta )$ on the probability margin $\delta$. 

5: We present our analysis in a modular way and obtain a complementary $\mathcal O (1 / T)$ convergence rate in the deterministic setting. 

6: To the best of our knowledge, this is the first high probability result for AdaGrad with a truly adaptive scheme, i.e., completely oblivious to the knowledge of smoothness and uniform variance bound, which simultaneously has best-known dependence of $\log( 1/ \delta)$. 

7: We further prove noise adaptation property of AdaGrad under additional noise assumptions.

8: \end{abstract}

9: