1: \begin{abstract}
2:
3: Automatically selecting the best performing algorithm for a given dataset or ranking multiple algorithms by their expected performance supports users in developing new machine learning applications. Most approaches for this problem rely on pre-computed dataset \meta and landmarking performances to capture the salient topology of the datasets and those topologies that the algorithms attend to. Landmarking usually exploits cheap algorithms not necessarily in the pool of candidate algorithms to get inexpensive approximations of the topology. While somewhat indicative, hand-crafted dataset \meta and landmarks are likely insufficient descriptors, strongly depending on the alignment of the geometries that the landmarks and the candidate algorithms search for. We propose \imfas, a method to exploit \mf landmarking information directly from the candidate algorithms in the form of non-parametrically non-myopic meta-learned learning curves via LSTMs in a few-shot setting during testing. Using this mechanism, \imfas jointly learns a dataset’s topology and algorithms’ inductive biases, without the need to expensively train them to convergence. Our approach produces informative landmarks, easily enriched by arbitrary meta-features at a low computational cost, capable of producing the desired ranking using cheaper fidelities. We additionally show that \imfas is able to beat \sh with at most 50\% of the fidelity sequence during test time.
4:
5: \end{abstract}
6: