BriefGPT.xyz
Jun, 2023
神经语言模型为何能解决下一个单词预测？数学角度分析
Why can neural language models solve next-word prediction? A mathematical perspective
HTML
PDF
Vinoth Nandakumar, Peng Mi, Tongliang Liu
TL;DR
本文研究一类可以用于模型英语句子的形式语言，证明神经语言模型可以在此背景下零误差地解决下一个单词预测任务，强调了嵌入层和完全连接部件在神经语言模型中的不同作用。
Abstract
Recently,
deep learning
has revolutionized the field of
natural language processing
, with
neural language models
proving to be very effect
→