It is well-known that document context is vital for resolving a range of
translation ambiguities, and in fact the document setting is the most natural
setting for nearly all translation. It is therefore unfortunate that machine
translation -- both research and production -- largely remains stuck in a
decades-old sentence-level translation paradigm. It is also an increasingly
glaring problem in light of competitive pressure from large language models,
which are natively document-based. Much work in document-context machine
translation exists, but for various reasons has been unable to catch hold. This
paper suggests a path out of this rut by addressing three impediments at once:
what architectures should we use? where do we get document-level information
for training them? and how do we know whether they are any good? In contrast to
work on specialized architectures, we show that the standard Transformer
architecture is sufficient, provided it has enough capacity. Next, we address
the training data issue by taking document samples from back-translated data
only, where the data is not only more readily available, but is also of higher
quality compared to parallel document data, which may contain machine
translation output. Finally, we propose generative variants of existing
contrastive metrics that are better able to discriminate among document
systems. Results in four large-data language pairs (DE$\rightarrow$EN,
EN$\rightarrow$DE, EN$\rightarrow$FR, and EN$\rightarrow$RU) establish the
success of these three pieces together in improving document-level performance.

本文提出了一种通过解决模型架构、训练数据和评估指标三个问题开展文本背景机器翻译的方案，并在 DE$ightarrow$EN，EN$ightarrow$DE，EN$ightarrow$FR 和 EN$ightarrow$RU 等语种的大数据实验中表明这三项改进措施能够显著提高文本背景翻译的性能。

机器翻译中逃离句子级范式

Escaping the sentence-level paradigm in machine translation

In this work, we study the effect of varying the architecture and training
data quality on the data scaling properties of Neural Machine Translation
(NMT). First, we establish that the test loss of encoder-decoder transformer
models scales as a power law in the number of training samples, with a
dependence on the model size. Then, we systematically vary aspects of the
training setup to understand how they impact the data scaling laws. In
particular, we change the following (1) Architecture and task setup: We compare
to a transformer-LSTM hybrid, and a decoder-only transformer with a language
modeling loss (2) Noise level in the training distribution: We experiment with
filtering, and adding iid synthetic noise. In all the above cases, we find that
the data scaling exponents are minimally impacted, suggesting that marginally
worse architectures or training data can be compensated for by adding more
data. Lastly, we find that using back-translated data instead of parallel data,
can significantly degrade the scaling exponent.

本文研究了神经机器翻译中体系结构和训练数据质量的变化对数据缩放性质的影响，并发现使用返向翻译数据会显著降低缩放系数。