Transformer encoders contextualize token representations by attending to all
other tokens at each layer, leading to quadratic increase in compute effort
with the input length. In practice, however, the input text of many NLP tasks
can be seen as a sequence of related segments (e.g., the sequence of sentences
within a passage, or the hypothesis and premise in NLI). While attending across
these segments is highly beneficial for many tasks, we hypothesize that this
interaction can be delayed until later encoding stages.
To this end, we introduce Layer-Adjustable Interactions in Transformers
(LAIT). Within LAIT, segmented inputs are first encoded independently, and then
jointly. This partial two-tower architecture bridges the gap between a Dual
Encoder's ability to pre-compute representations for segments and a fully
self-attentive Transformer's capacity to model cross-segment attention. The
LAIT framework effectively leverages existing pretrained Transformers and
converts them into the hybrid of the two aforementioned architectures, allowing
for easy and intuitive control over the performance-efficiency tradeoff.
Experimenting on a wide range of NLP tasks, we find LAIT able to reduce 30-50%
of the attention FLOPs on many tasks, while preserving high accuracy; in some
practical settings, LAIT could reduce actual latency by orders of magnitude.

本研究提出了一种名为 “Layer-Adjustable Interactions” 的编码器架构，可将输入的文本分成片段进行单独编码和联合编码，以实现在提高多项自然语言处理任务的表现的同时，减少注意力计算量。