BriefGPT.xyz
Apr, 2024
像Transformer一样进行计数：将时间计数逻辑编译成Softmax Transformers
Counting Like Transformers: Compiling Temporal Counting Logic Into Softmax Transformers
HTML
PDF
Andy Yang, David Chiang
TL;DR
基于序列变换器的计算能力，提出了时序计数逻辑Kt和C-RASP变种，并证明它们可以编译为具有未限制输入大小的未来掩码软注意力变换器，从而形成了迄今为止已知的形式表达能力下界。
Abstract
Deriving formal bounds on the
expressivity
of
transformers
, as well as studying
transformers
that are constructed to implement known algor
→