Multiple business scenarios require an automated generation of descriptive
human-readable text from structured input data. Hence, fact-to-text generation
systems have been developed for various downstream tasks like generating soccer
reports, weather and financial reports, medical reports, person biographies,
etc. Unfortunately, previous work on fact-to-text (F2T) generation has focused
primarily on English mainly due to the high availability of relevant datasets.
Only recently, the problem of cross-lingual fact-to-text (XF2T) was proposed
for generation across multiple languages alongwith a dataset, XALIGN for eight
languages. However, there has been no rigorous work on the actual XF2T
generation problem. We extend XALIGN dataset with annotated data for four more
languages: Punjabi, Malayalam, Assamese and Oriya. We conduct an extensive
study using popular Transformer-based text generation models on our extended
multi-lingual dataset, which we call XALIGNV2. Further, we investigate the
performance of different text generation strategies: multiple variations of
pretraining, fact-aware embeddings and structure-aware input encoding. Our
extensive experiments show that a multi-lingual mT5 model which uses fact-aware
embeddings with structure-aware input encoding leads to best results on average
across the twelve languages. We make our code, dataset and model publicly
available, and hope that this will help advance further research in this
critical area.

本论文提出了跨语言机器自动事实到文本生成的方法，使用包括 BERT 和 GPT-2 在内的 Transformer-based 模型和多语言数据集 XALIGNV2，得到了优秀的结果。

XF2T: 跨语言低资源事实到文本生成

XF2T: Cross-lingual Fact-to-Text Generation for Low-Resource Languages

In this paper, we explore the challenging problem of performing a generative
task in a target language when labeled data is only available in English, using
summarization as a case study. We assume a strict setting with no access to
parallel data or machine translation and find that common transfer learning
approaches struggle in this setting, as a generative multilingual model
fine-tuned purely on English catastrophically forgets how to generate
non-English. Given the recent rise of parameter-efficient adaptation
techniques, we conduct the first investigation into how one such method, prompt
tuning (Lester et al., 2021), can overcome catastrophic forgetting to enable
zero-shot cross-lingual generation. Our experiments show that
parameter-efficient prompt tuning provides gains over standard fine-tuning when
transferring between less-related languages, e.g., from English to Thai.
However, a significant gap still remains between these methods and
fully-supervised baselines. To improve cross-lingual transfer further, we
explore several approaches, including: (1) mixing in unlabeled multilingual
data, and (2) explicitly factoring prompts into recombinable language and task
components. Our approaches can provide further quality gains, suggesting that
robust zero-shot cross-lingual generation is within reach.

本文研究如何在只有英文标记数据的情况下，在目标语言中执行生成任务，以 summarization 作为案例研究。研究发现了通用的迁移学习方法在这种情况下存在困难，并且展示通过 prompt tuning 等方法可以实现跨语言生成。