BriefGPT.xyz
Aug, 2023
通过预条件器解决关注度核回归问题
Solving Attention Kernel Regression Problem via Pre-conditioner
HTML
PDF
Zhao Song, Junze Yin, Lichen Zhang
TL;DR
通过计算注意力矩阵，大型语言模型在许多任务中展现出了令人印象深刻的性能。本研究定义和研究了一种新问题，即注意力内核回归问题，并展示了如何在数据矩阵的输入稀疏时间内解决该问题。
Abstract
large language models
have shown impressive performance in many tasks. One of the major features from the
computation
perspective is computing the
→