Existing work in document-level neural machine translation commonly
concatenates several consecutive sentences as a pseudo-document, and then
learns inter-sentential dependencies. This strategy limits the model's ability
to leverage information from distant context. We overcome this limitation with
a novel Document Flattening (DocFlat) technique that integrates Flat-Batch
Attention (FBA) and Neural Context Gate (NCG) into Transformer model to utilize
information beyond the pseudo-document boundaries. FBA allows the model to
attend to all the positions in the batch and learns the relationships between
positions explicitly and NCG identifies the useful information from the distant
context. We conduct comprehensive experiments and analyses on three benchmark
datasets for English-German translation, and validate the effectiveness of two
variants of DocFlat. Empirical results show that our approach outperforms
strong baselines with statistical significance on BLEU, COMET and accuracy on
the contrastive test set. The analyses highlight that DocFlat is highly
effective in capturing the long-range information.

本文提出了一种文档展开技术（DocFlat），通过整合扁平批量注意力（FBA）和神经上下文门（NCG）到 Transformer 模型中，使其能够利用超出伪文档边界的信息，实现捕捉长距离信息，并在三个基准数据集上进行了全面实验和分析，验证了 DocFlat 及其变体的有效性，并显示出较强的性能。