In data-to-text (D2T) generation, training on in-domain data leads to
overfitting to the data representation and repeating training data noise. We
examine how to avoid finetuning pretrained language models (PLMs) on D2T
generation datasets while still taking advantage of surface realization
capabilities of PLMs. Inspired by pipeline approaches, we propose to generate
text by transforming single-item descriptions with a sequence of modules
trained on general-domain text-based operations: ordering, aggregation, and
paragraph compression. We train PLMs for performing these operations on a
synthetic corpus WikiFluent which we build from English Wikipedia. Our
experiments on two major triple-to-text datasets -- WebNLG and E2E -- show that
our approach enables D2T generation from RDF triples in zero-shot settings.

本文利用预处理语言模型进行通用领域训练，使用由维基百科构建的 WikiFluent 数据集，在 WebNLG 和 E2E 数据集上进行实验证明，使用单个项目描述生成 D2T 是可行的，且可以从 RDF 三元组中完成零 - shot D2T 生成。

零样本数据到文本生成的神经管线

Neural Pipeline for Zero-Shot Data-to-Text Generation

Recent progress in language model pre-training has achieved a great success
via leveraging large-scale unstructured textual data. However, it is still a
challenge to apply pre-training on structured tabular data due to the absence
of large-scale high-quality tabular data. In this paper, we propose TAPEX to
show that table pre-training can be achieved by learning a neural SQL executor
over a synthetic corpus, which is obtained by automatically synthesizing
executable SQL queries and their execution outputs. TAPEX addresses the data
scarcity challenge via guiding the language model to mimic a SQL executor on
the diverse, large-scale and high-quality synthetic corpus. We evaluate TAPEX
on four benchmark datasets. Experimental results demonstrate that TAPEX
outperforms previous table pre-training approaches by a large margin and
achieves new state-of-the-art results on all of them. This includes the
improvements on the weakly-supervised WikiSQL denotation accuracy to 89.5%
(+2.3%), the WikiTableQuestions denotation accuracy to 57.5% (+4.8%), the SQA
denotation accuracy to 74.5% (+3.5%), and the TabFact accuracy to 84.2%
(+3.2%). To our knowledge, this is the first work to exploit table pre-training
via synthetic executable programs and to achieve new state-of-the-art results
on various downstream tasks. Our code can be found at
this https URL

本文提出了 TAPEX，利用合成语料库学习神经 SQL 执行器来解决表格预训练中数据稀缺的挑战，并在四个基准数据集上实现了新的最先进结果。

TAPEX: 通过学习神经 SQL 执行器进行表格预训练

TAPEX: Table Pre-training via Learning a Neural SQL Executor

This paper describes NTT's submission to the WMT19 robustness task. This task
mainly focuses on translating noisy text (e.g., posts on Twitter), which
presents different difficulties from typical translation tasks such as news.
Our submission combined techniques including utilization of a synthetic corpus,
domain adaptation, and a placeholder mechanism, which significantly improved
over the previous baseline. Experimental results revealed the placeholder
mechanism, which temporarily replaces the non-standard tokens including emojis
and emoticons with special placeholder tokens during translation, improves
translation accuracy even with noisy texts.

本文介绍 NTT 的 WMT19 鲁棒性任务中的提交成果，并使用合成语料库、领域自适应和占位符机制等技术，大幅优于以前的基线。实验结果显示，占位符机制可以在翻译过程中临时替换包括表情符号等非标准标记，即使在处理带噪音的文本时也能提高翻译精度。

NTT 关于 WMT19 健壮性任务的机器翻译系统

NTT's Machine Translation Systems for WMT19 Robustness Task

Training models for the automatic correction of machine-translated text
usually relies on data consisting of (source, MT, human post- edit) triplets
providing, for each source sentence, examples of translation errors with the
corresponding corrections made by a human post-editor. Ideally, a large amount
of data of this kind should allow the model to learn reliable correction
patterns and effectively apply them at test stage on unseen (source, MT) pairs.
In practice, however, their limited availability calls for solutions that also
integrate in the training process other sources of knowledge. Along this
direction, state-of-the-art results have been recently achieved by systems
that, in addition to a limited amount of available training data, exploit
artificial corpora that approximate elements of the "gold" training instances
with automatic translations. Following this idea, we present eSCAPE, the
largest freely-available Synthetic Corpus for Automatic Post-Editing released
so far. eSCAPE consists of millions of entries in which the MT element of the
training triplets has been obtained by translating the source side of
publicly-available parallel corpora, and using the target side as an artificial
human post-edit. Translations are obtained both with phrase-based and neural
models. For each MT paradigm, eSCAPE contains 7.2 million triplets for
English-German and 3.3 millions for English-Italian, resulting in a total of
14,4 and 6,6 million instances respectively. The usefulness of eSCAPE is proved
through experiments in a general-domain scenario, the most challenging one for
automatic post-editing. For both language directions, the models trained on our
artificial data always improve MT quality with statistically significant gains.
The current version of eSCAPE can be freely downloaded from:
this http URL

该论文介绍了 eSCAPE，这是目前最大的免费合成语料库，为机器翻译的自动后编辑训练模型提供了大量数据，并使用模型在通用领域方案中实验证明了其有效性。