1: \begin{abstract}

2: A recent line of ground-breaking results for permutation-based SGD  has corroborated a widely observed phenomenon: random permutations offer faster convergence than with-replacement sampling. 

3: However, is random optimal?

4: We show that this depends heavily on what functions we are optimizing, and the convergence gap between optimal and random permutations can vary from exponential to nonexistent.

5: We first show that for 1-dimensional strongly convex functions, with smooth second derivatives, there exist 

6: permutations that offer exponentially faster convergence compared to random.

7: However, for general strongly convex functions, random permutations are optimal.

8: Finally, we show that for quadratic, strongly-convex functions, there are easy-to-construct permutations that lead to accelerated convergence compared to random.

9: Our results suggest that a general convergence characterization of optimal permutations cannot capture the nuances of individual function classes, and can  mistakenly indicate that one cannot do much better than random.

10: \end{abstract}

11: