BriefGPT.xyz
May, 2024
战略数据排序: 通过课程学习提升大型语言模型性能
Strategic Data Ordering: Enhancing Large Language Model Performance through Curriculum Learning
HTML
PDF
Jisu Kim, Juhwan Lee
TL;DR
通过课程学习的数据中心培训策略,根据数据的不同指标进行排序可以提高大型语言模型的性能,而无需增加模型大小或数据集容量,从而解决大型语言模型培训中的可扩展性挑战。
Abstract
The rapid advancement of
large language models
(LLMs) has improved text understanding and generation but poses challenges in computational resources. This study proposes a
curriculum learning
-inspired,
→