1: \begin{abstract}
2:
3: We present global
4: convergence rates for a line-search method which is based on random
5: first-order models and directions whose
6: quality is ensured only with certain probability. We show that in terms of the order of the accuracy, the evaluation
7: complexity of such a method is the same as its counterparts that use
8: deterministic accurate models; the use of
9: probabilistic models only increases the complexity by a constant, which depends on the probability of the models being good.
10: We particularize and improve these results in the convex and strongly convex case.
11:
12: We also analyze a probabilistic cubic regularization
13: variant that allows approximate probabilistic second-order models and show improved complexity bounds
14: compared to probabilistic first-order methods; again, as a function of the accuracy, the probabilistic cubic regularization
15: bounds are of the same (optimal) order as for the deterministic case.
16:
17: %Line-search methods are an important class of algorithms for
18: %unconstrained optimization which rely on approximately computing a local descent direction
19: %and computing a step along this direction in such a way that a sufficient decrease is achieved.
20: %To ensure that sufficient decrease is possible the direction has to satisfy certain requirements.
21:
22: %Often, in practical applications, ensuring that the direction properties are always what is needed is
23: %prohibitively expensive or impossible. This may be due to the fact that derivative information about the objective function is not available
24: %or because full gradient (and Hessian) are too expensive to compute, or a model of the true objective function is too expensive to optimize accurately.
25:
26: %In this paper we present the analysis of a line-search method which is based on random models and direction whose quality is ensured only with certain probability.
27: \end{abstract}
28: