ff6dbcce88b73d84.tex
1: \begin{abstract}
2: 
3: 
4: 
5: Preconditioning is a crucial operation in gradient-based numerical optimisation. It helps decrease the local condition number of a function by appropriately transforming its gradient. For a convex function, where the gradient can be computed exactly, the optimal linear transformation corresponds to the inverse of the Hessian operator, while the optimal convex transformation is the convex conjugate of the function. 
6: Different conditions result in variations of these dependencies. Practical algorithms often employ low-rank or stochastic approximations of the inverse Hessian matrix for preconditioning. However, theoretical guarantees for these algorithms typically lack a justification for the defining property of preconditioning. 
7: %    Preconditioning, in gradient-based numerical optimisation, is operation which allows the local condition number of the function to be decreased by the proper transformation of the gradient. If the function is convex and gradient can be computed exactly, the optimal linear transformation is inverse to the Hessian operator, the optimal convex one~--- is a convex conjugate of the function; other conditions lead to variations of same dependencies. In practical algorithms, low-rank or stochastic approximation of inverse Hessian matrix is used for preconditioning, but theoretical guarantees for these algorithms usually miss the justification of defining property of preconditioning.
8:     This paper presents a simple theoretical framework that demonstrates, given a smooth function and an available unbiased stochastic approximation of its gradient, that it is possible to refine the dependency of the convergence rate on the Lipschitz constant of the gradient.
9:     
10: %    In this paper, simple theoretical framework is used to demonstrate, if function is smooth and unbiased stochastic approximation of gradient is available, that the dependency of convergence rate on Lipschitz constant of the gradient can be refined.
11: \end{abstract}
12: