With the rise of deep learning, large datasets and complex models have become
common, requiring significant computing power. To address this, data
distillation has emerged as a technique to quickly train models with lower
memory and time requirements. However, data distillation on text-based datasets
hasn't been explored much because of the challenges rising due to its discrete
nature. Additionally, existing dataset distillation methods often struggle to
generalize to new architectures. In the paper, we propose several data
distillation techniques for multilingual text classification datasets using
language-model-based learning methods. We conduct experiments to analyze their
performance in terms of classification strength, and cross-architecture
generalization. Furthermore, we investigate the language-specific fairness of
the data summaries generated by these methods. Our approach builds upon
existing techniques, enhancing cross-architecture generalization in the text
data distillation domain.

通过使用基于语言模型的学习方法，我们在多语言文本分类数据集上提出了几种数据蒸馏技术，来增强文本数据蒸馏领域中的跨体系结构泛化能力，并分析了它们在分类强度和跨体系结构泛化方面的性能，也研究了这些方法生成的数据摘要的语言特定公平性。