1: \begin{abstract}

2: Conventional Bayesian optimization relies on handcrafted acquisition functions (AFs) to sequentially determine the sample points. However, it has been widely observed in practice that the best-performing AF can vary significantly under different ground-truth black-box functions. It has remained as a challenge to design one AF that can attain the best performance over a wide variety of black-box functions. This paper aims to attack this challenge through the perspective of reinforced few-shot learning for AFs. Specifically, we first connect the notion of AFs with Q-functions and present a Bayesian variant of deep Q-learning that learns a distribution of Q-networks as AFs based on the Kullback-Leibler (KL) regularization framework. This design inherently provides the uncertainty required in sampling for Bayesian optimization and mitigates the potential overfitting. On the meta-level, we leverage the Bayesian model-agnostic meta-learning, which serves as a natural companion to the proposed Q-based AF in the few-shot setting. Through extensive experiments, we demonstrate that the proposed few-shot acquisition function indeed achieves comparable or better regret performance than the state-of-the-art benchmarks on a wide variety of synthetic and real datasets.

3: %Specifically, we leverage the Bayesian variant of deep Q-learning model-agnostic meta-learning (MAML) to construct a single neural acquisition function that can easily adapt to various black-box functions with only a small number of samples. Given that MAML is known to suffer from training instability, we propose a series of architectural design to address the convergence issue. 

4: \end{abstract}

5: