In the rapidly advancing arena of large language models (LLMs), a key
challenge is to enhance their capabilities amid a looming shortage of
high-quality training data. Our study starts from an empirical strategy for the
light continual training of LLMs using their original pre-training data sets,
with a specific focus on selective retention of samples that incur moderately
high losses. These samples are deemed informative and beneficial for model
refinement, contrasting with the highest-loss samples, which would be discarded
due to their correlation with data noise and complexity. We then formalize this
strategy into a principled framework of Instance-Reweighted Distributionally
Robust Optimization (IR-DRO). IR-DRO is designed to dynamically prioritize the
training focus on informative samples through an instance reweighting
mechanism, streamlined by a closed-form solution for straightforward
integration into established training protocols. Through rigorous
experimentation with various models and datasets, our findings indicate that
our sample-targeted methods significantly improve LLM performance across
multiple benchmarks, in both continual pre-training and instruction tuning
scenarios. Our codes are available at
this https URL

使用实例重新加权的分布鲁棒优化方法在大型语言模型的持续训练中有效地优化了样本保留，提高了语言模型的性能。