Document-level machine translation leverages inter-sentence dependencies to
produce more coherent and consistent translations. However, these models,
predominantly based on transformers, are difficult to scale to long documents
as their attention layers have quadratic complexity in the sequence length.
Recent efforts on efficient attention improve scalability, but their effect on
document translation remains unexplored. In this work, we investigate the
efficacy of a recent linear attention model by Peng et al. (2021) on document
translation and augment it with a sentential gate to promote a recency
inductive bias. We evaluate the model on IWSLT 2015 and OpenSubtitles 2018
against the transformer, demonstrating substantially increased decoding speed
on long sequences with similar or better BLEU scores. We show that sentential
gating further improves translation quality on IWSLT.

本文研究了一种基于 Peng et al. (2021) 的线性注意力模型，并通过句子门促进最近归纳偏置，以提高文档级机器翻译的效能和翻译质量，并在 IWSLT 2015 和 OpenSubtitles 2018 上进行了评估和测试，结果表明该模型在长序列的解码速度方面有大幅提高，且翻译得分相似或更高，并展示了句子门进一步提高了 IWSLT 上的翻译质量。