1: \begin{abstract}

2: We propose a new class of rigorous methods for derivative-free optimization with the aim of delivering efficient and robust numerical performance for functions of all types, from smooth to non-smooth, and under different noise regimes. To this end, we have developed a class

3: of methods, called \FLE{} methods, organized around two main types of iterations.

4: 

5: The first iteration type (called \FullEval{}) is expensive in function evaluations, but exhibits good performance in the smooth and non-noisy cases. For the theory, we consider a line search based on an approximate gradient, backtracking until a sufficient decrease condition is satisfied. In practice, the gradient was approximated via finite differences, and the direction was calculated by a quasi-Newton step (BFGS). The second iteration type (called \LowEval{}) is cheap in function evaluations, yet more robust in the presence of noise or non-smoothness. For the theory, we consider direct search, and in practice we use probabilistic direct search with one random direction and its negative.

6: 

7: A switch condition from \FullEval{} to \LowEval{} iterations was developed based on the values of the line-search and direct-search stepsizes. If enough \FullEval{} steps are taken, we derive a complexity result of gradient-descent type. Under failure of \FullEval{}, the \LowEval{} iterations become the drivers of convergence yielding non-smooth convergence. \FLE{} methods are shown to be efficient and robust in practice across problems with different levels of smoothness and noise.

8: \end{abstract}

9: