BriefGPT.xyz
Feb, 2021
Transformers中的位置信息: 概述
Position Information in Transformers: An Overview
HTML
PDF
Philipp Dufter, Martin Schmitt, Hinrich Schütze
TL;DR
本文综述了如何将位置信息整合到Transformer模型中，以比较不同方法在重要模型维度上的优劣，以及指出在选择位置编码时应考虑的应用特性，并为未来的研究提供刺激。
Abstract
transformers
are arguably the main workhorse in recent
natural language processing
research. By definition a Transformer is invariant with respect to reorderings of the input. However, language is inherently sequ
→