1: \begin{abstract}

2: The great success of modern machine learning models on large datasets is contingent on extensive computational resources with high financial and environmental costs. One way to address this is by extracting subsets that generalize on par with the full data. In this work, we propose a general framework, \model{}, which finds subsets that closely match the gradient of the \emph{training or validation} set. We find such subsets effectively using an orthogonal matching pursuit algorithm. We show rigorous theoretical and convergence guarantees of the proposed algorithm and, through our extensive experiments on real-world datasets, show the effectiveness of our proposed framework. We show that \textsc{Grad-Match} significantly and consistently outperforms several recent data-selection algorithms and achieves the best accuracy-efficiency trade-off. \textsc{Grad-Match} is available as a part of the CORDS toolkit: \url{https://github.com/decile-team/cords}.

3: %\footnote{\scriptsize{The code of \model{} is available at \url{https://anonymous.4open.science/r/2eada7f6-5514-4409-8d90-ff4f8d7d5c03/}}}.

4: %For example, when training with ResNet-18 on CIFAR-10, we observe a 7x and 3x speedup (and similar energy saving) with an accuracy drop of 2.5\% and 0.8\% respectively, while for CIFAR-100, we achieve a 4.8x and 3x speedup and energy saving, while loosing around 2.1\% and 0.7\% in accuracy.\looseness-1

5: \end{abstract}

6: