Data availability limits the scope of any given task. In machine translation,
historical models were incapable of handling longer contexts, so the lack of
document-level datasets was less noticeable. Now, despite the emergence of
long-sequence methods, we remain within a sentence-level paradigm and without
data to adequately approach context-aware machine translation. Most large-scale
datasets have been processed through a pipeline that discards document-level
metadata. In this work, we reconstruct document-level information for three
(ParaCrawl, News Commentary, and Europarl) large datasets in German, French,
Spanish, Italian, Polish, and Portuguese (paired with English). We then
introduce a document-level filtering technique as an alternative to traditional
bitext filtering. We present this filtering with analysis to show that this
method prefers context-consistent translations rather than those that may have
been sentence-level machine translated. Last we train models on these longer
contexts and demonstrate improvement in document-level translation without
degradation of sentence-level translation. We release our dataset, ParaDocs,
and resulting models as a resource to the community.

我们对德语、法语、西班牙语、意大利语、波兰语和葡萄牙语的三个大规模数据集（ParaCrawl、新闻评论和 Europarl）进行了文档级信息的重建，并引入了文档级过滤技术作为传统双语过滤的替代方法。我们通过分析表明，该方法偏好于与上下文一致的翻译，而不是可能是句级机器翻译的翻译。最后，我们在这些更长的上下文上训练模型，并证明了文档级翻译的改进而不降低句级翻译。我们将我们的数据集 ParaDocs 和生成的模型作为资源发布给社区。