1: \begin{abstract}

2: Modern stochastic optimization methods often rely on uniform sampling which is agnostic to the underlying characteristics of the data.

3: This might degrade the convergence by  yielding estimates that suffer from a high variance.  

4: A possible remedy is to employ non-uniform \emph{importance sampling} techniques, which take the structure of the dataset into account. 

5: In this work, we investigate a recently proposed  setting  which poses variance reduction as an online optimization problem with bandit feedback.

6: We devise a novel and efficient algorithm for this setting  that finds a sequence of importance sampling distributions competitive with the best fixed distribution in hindsight, the first result of this kind.

7: While we present our method for sampling datapoints, it naturally extends to selecting coordinates or even blocks of thereof.  Empirical validations underline the benefits of our method in several settings.

8: \end{abstract}

9: