1: \begin{abstract}

2: Hyper-kernels endowed by hyper-Reproducing Kernel Hilbert Space (hyper-RKHS) formulate the kernel learning task as learning on the space of kernels itself, which provides significant model flexibility for kernel learning with outstanding performance in real-world applications.

3: However, the convergence behavior of these learning algorithms in hyper-RKHS has not been investigated in learning theory.

4: In this paper, we conduct approximation analysis of kernel ridge regression (KRR) and support vector regression (SVR) in this space.

5: To the best of our knowledge,  this is the first work to study the approximation performance of regression in hyper-RKHS.

6: For applications, we propose a general kernel learning framework conducted by the introduced two regression models to deal with the out-of-sample extensions problem, \emph{i.e.}, to learn a underlying general kernel from the pre-given kernel/similarity matrix in hyper-RKHS.

7: %Accordingly, the out-of-sample issue is formulated as a regression problem, which can be solved by the derived KRR and SVR in this space.

8: %The estimated learning rate in this space is better than that of SVR in the traditional Reproducing Kernel Hilbert Spaces (RKHS).

9: Experimental results on several benchmark datasets suggest that our methods are able to learn a general kernel function from an arbitrary given kernel matrix.

10: %Experimental results on several benchmarks suggest that the proposed  method is able to learn a kernel function form an arbitrary kernel matrix, and thus achieves a satisfactory performance on classification tasks.

11: \end{abstract}

12: