1: \begin{abstract}

2:     We consider the problem of asynchronous stochastic optimization, where

3:     an optimization algorithm makes updates based on stale stochastic gradients of the objective that are subject to an arbitrary (possibly adversarial) sequence of delays.

4:     We present a procedure which, for any given $q \in (0,1]$, transforms any standard stochastic first-order method to an asynchronous method with convergence guarantee depending on the $q$-quantile delay of the sequence.

5:     This approach leads to convergence rates of the form $O(\delayqq/qT+\sigma/\sqrt{qT})$ for non-convex and $O(\delayqq^2/(q T)^2+\sigma/\sqrt{qT})$ for convex smooth problems, where $\delayqq$ is the $q$-quantile delay, generalizing and improving on existing results that depend on the average delay.

6:     We further show a method that 

7:     automatically adapts to all quantiles simultaneously, without any prior knowledge of the delays, achieving convergence rates of the form $O(\inf_{q} \delayqq/qT+\sigma/\sqrt{qT})$ for non-convex and $O(\inf_{q} \delayqq^2/(q T)^2+\sigma/\sqrt{qT})$ for convex smooth problems.

8:     Our technique is based on asynchronous mini-batching with a careful batch-size selection and filtering of stale gradients.

9: \end{abstract}

10: