The great success of modern machine learning models on large datasets is
contingent on extensive computational resources with high financial and
environmental costs. One way to address this is by extracting subsets that
generalize on par with the full data. In this work, we propose a general
framework, GRAD-MATCH, which finds subsets that closely match the gradient of
the training or validation set. We find such subsets effectively using an
orthogonal matching pursuit algorithm. We show rigorous theoretical and
convergence guarantees of the proposed algorithm and, through our extensive
experiments on real-world datasets, show the effectiveness of our proposed
framework. We show that GRAD-MATCH significantly and consistently outperforms
several recent data-selection algorithms and achieves the best
accuracy-efficiency trade-off. GRAD-MATCH is available as a part of the CORDS
toolkit: https://github.com/decile-team/cords.

本文提出了 GRAD-MATCH 作为一种有效的数据子集提取算法，该算法使用正交匹配追踪算法有效地找到与训练或验证集的梯度相匹配的数据子集，实验表明该算法在精度和效率上都有显著提升。