Dataset pruning aims to select a subset of a dataset for efficient model training. While data efficiency in natural language processing has primarily focused on within-corpus scenarios during model pre-training, efficient dataset pruning for task-specific fine-tuning across diverse datasets remains challenging due to variability in dataset sizes, data distributions, class imbalance and label spaces. Current cross-dataset pruning techniques for fine-tuning often rely on computationally expensive sample ranking processes, typically requiring full dataset training or reference models. We address this gap by proposing Swift Cross-Dataset Pruning (SCDP). Specifically, our approach uses TF-IDF embeddings with geometric median to rapidly evaluate sample importance. We then apply dataset size-adaptive pruning to ensure diversity: for smaller datasets, we retain samples far from the geometric median, while for larger ones, we employ distance-based stratified pruning. Experimental results on six diverse datasets demonstrate the effectiveness of our method, spanning various tasks and scales while significantly reducing computational resources. Source code is available at: https://github.com/he-y/NLP-Dataset-Pruning

本研究解决了在多样数据集上进行任务特定微调时，数据集剪枝效率低下的问题。我们提出的快速跨数据集剪枝（SCDP）方法，通过使用TF-IDF嵌入和几何中位数快速评估样本重要性，并根据数据集大小进行适应性剪枝，显著减少了计算资源消耗。实验结果表明，该方法在多个不同任务和规模的数据集上表现出色。

快速跨数据集剪枝：提高自然语言理解中的微调效率