Data-to-text generation has recently attracted substantial interests due to
its wide applications. Existing methods have shown impressive performance on an
array of tasks. However, they rely on a significant amount of labeled data for
each task, which is costly to acquire and thus limits their application to new
tasks and domains. In this paper, we propose to leverage pre-training and
transfer learning to address this issue. We propose a knowledge-grounded
pre-training (KGPT), which consists of two parts, 1) a general
knowledge-grounded generation model to generate knowledge-enriched text. 2) a
pre-training paradigm on a massive knowledge-grounded text corpus crawled from
the web. The pre-trained model can be fine-tuned on various data-to-text
generation tasks to generate task-specific text. We adopt three settings,
namely fully-supervised, zero-shot, few-shot to evaluate its effectiveness.
Under the fully-supervised setting, our model can achieve remarkable gains over
the known baselines. Under zero-shot setting, our model without seeing any
examples achieves over 30 ROUGE-L on WebNLG while all other baselines fail.
Under the few-shot setting, our model only needs about one-fifteenth as many
labeled examples to achieve the same level of performance as baseline models.
These experiments consistently prove the strong generalization ability of our
proposed framework this https URL

本文提出一种使用预训练和迁移学习处理数据生成文本任务的方法，该方法包括知识驱动的预训练模型和针对各种任务生成文本的微调模型，其在少样本情况下性能优异，验证了其强大的泛化能力。