1: \begin{abstract}

2: Bayesian optimization is a sample-efficient method for finding a global optimum 

3: of an expensive-to-evaluate black-box function.

4: A global solution is found by accumulating a pair of query point and its function value,

5: repeating these two procedures:

6: (i) modeling a surrogate function;

7: (ii) maximizing an acquisition function to determine where next to query.

8: Convergence guarantees are only valid when the global optimizer 

9: of the acquisition function is found at each round

10: and selected as the next query point.

11: In practice, however, local optimizers of an acquisition function are also used, 

12: since searching for the global optimizer is often a non-trivial or time-consuming task.

13: In this paper we consider three popular acquisition functions, PI, EI, and GP-UCB 

14: induced by Gaussian process regression. 

15: Then we present a performance analysis on the behavior of local optimizers 

16: of those acquisition functions, in terms of {\em instantaneous regrets} over global optimizers.

17: We also introduce an analysis, 

18: allowing a local optimization method to start from multiple different initial conditions.

19: Numerical experiments confirm the validity of our theoretical analysis.

20: 

21: \keywords{Global optimization \and Bayesian optimization \and Acquisition function optimization \and Instantaneous regret analysis.}

22: \end{abstract}

23: