BriefGPT.xyz
May, 2023
不牺牲语言熟练度的情况下学习非语言技能
Learning Non-linguistic Skills without Sacrificing Linguistic Proficiency
HTML
PDF
Mandar Sharma, Nikhil Muralidhar, Naren Ramakrishnan
TL;DR
本文提出了一种基于信息论干预和特定技能损失的新型非语言技能注入框架,可使LLMs学习严格的算术推理,相比注入非语言技能和保持语言知识的现有技术,我们的模型在使用少量数据且不产生额外合成语言训练数据的情况下表现更好。
Abstract
The field of
math-nlp
has witnessed significant growth in recent years, motivated by the desire to expand LLM performance to the learning of non-linguistic notions (numerals, and subsequently,
arithmetic reasoning
→