1: \begin{abstract}%   <- trailing '%' for backward compatibility of .sty file

2: 	The zeroth-order optimization has been widely used in machine learning applications. However, the theoretical study of the zeroth-order optimization  focus on the algorithms which approximate (first-order) gradients using (zeroth-order) function value difference at a random direction. 

3: 	The theory of algorithms which  approximate the gradient and Hessian information by zeroth-order queries is much less studied.

4: 	In this paper, we focus on the theory of zeroth-order optimization which utilizes both the first-order and second-order information approximated by the zeroth-order queries.

5: 	We first propose a novel reparameterized objective function with parameters $(\mu, \Sigma)$. 

6: 	This reparameterized objective function achieves its optimum at the minimizer and the Hessian inverse of the original objective function respectively, but with small perturbations.

7: 	Accordingly, we propose a new algorithm to minimize our proposed reparameterized objective, which we call \texttt{MiNES} (mirror descent natural evolution strategy).

8: 	We show that the estimated covariance matrix of \texttt{MiNES} converges to the inverse of Hessian  matrix of the objective function with a  convergence rate $\widetilde{\cO}(1/k)$, where $k$ is the iteration number and $\widetilde{\cO}(\cdot)$ hides the constant and $\log$ terms.

9: 	We also provide the explict convergence rate of \texttt{MiNES} and how the covariance matrix promote the convergence rate.

10: \end{abstract}

11: