BriefGPT.xyz
May, 2024
循环神经网络语言模型表达能力下界
Lower Bounds on the Expressivity of Recurrent Neural Language Models
HTML
PDF
Anej Svete, Franz Nowak, Anisha Mohamed Sahabdeen, Ryan Cotterell
TL;DR
通过将递归神经网络语言模型连接到概率有限状态自动机，我们重新审视了递归神经网络语言模型的表征能力，并证明具有线性边界精度的递归神经网络语言模型可以表示任意的正则语言模型。
Abstract
The recent successes and spread of large
neural language models
(LMs) call for a thorough understanding of their computational ability. Describing their computational abilities through LMs' \emph{
representational capaci
→