1: \begin{abstract}

2:     The kernel-based regularization method has two core issues: kernel

3:     design and hyperparameter estimation. In this paper, we focus on the second issue and study the

4:     properties of several hyperparameter estimators including the

5:     empirical Bayes (EB) estimator, two Stein's unbiased risk

6:     estimators (SURE) and their corresponding Oracle counterparts, with an emphasis on the asymptotic properties of these hyperparameter estimators. To this goal, we first derive and

7:     then rewrite the first order optimality conditions of these hyperparameter estimators, leading to several insights on these

8:     hyperparameter estimators. Then we show that as the number of data

9:     goes to infinity, the two SUREs converge to the best hyperparameter

10:     minimizing the corresponding mean square error, respectively, while

11:     the more widely used EB estimator converges to another best

12:     hyperparameter minimizing the expectation of the EB estimation criterion. This indicates that

13:     the two SUREs are asymptotically optimal but the EB estimator is

14:     not. Surprisingly, the convergence rate of two SUREs is slower than

15:     that of the EB estimator, and moreover, unlike the two SUREs, the EB

16:     estimator is independent of the convergence rate of $\Phi^T\Phi/N$ to

17:     its limit, where $\Phi$ is the regression matrix and $N$ is the

18:     number of data. A Monte Carlo simulation is provided to demonstrate

19:     the theoretical results.

20: \end{abstract}