Although remarkable progress on the neural table-to-text methods has been
made, the generalization issues hinder the applicability of these models due to
the limited source tables. Large-scale pretrained language models sound like a
promising solution to tackle such issues. However, how to effectively bridge
the gap between the structured table and the text input by fully leveraging
table information to fuel the pretrained model is still not well explored.
Besides, another challenge of integrating the deliberation mechanism into the
text-to-text pretrained model for solving the table-to-text task remains seldom
studied. In this paper, to implement the table-to-text generation with
pretrained language model, we propose a table structure understanding and text
deliberating approach, namely TASD. Specifically, we devise a three-layered
multi-head attention network to realize the table-structure-aware text
generation model with the help of the pretrained language model. Furthermore, a
multi-pass decoder framework is adopted to enhance the capability of polishing
generated text for table descriptions. The empirical studies, as well as human
evaluation, on two public datasets, validate that our approach can generate
faithful and fluent descriptive texts for different types of tables.

本文提出了一种名为 TASD 的表结构理解和文本判断方法，采用三层多头自注意力网络实现了以预训练语言模型为基础的文本生成模型，并采用多遍解码器框架增强了表格描述的生成能力，实验证明该方法可以为不同类型的表格生成准确流畅的文本描述。

利用预训练语言模型实现表格到文本的生成：一种表格结构理解和文本构思的方法

Towards Table-to-Text Generation with Pretrained Language Model: A Table Structure Understanding and Text Deliberating Approach

In this work, we focus on the task of generating natural language
descriptions from a structured table of facts containing fields (such as
nationality, occupation, etc) and values (such as Indian, actor, director,
etc). One simple choice is to treat the table as a sequence of fields and
values and then use a standard seq2seq model for this task. However, such a
model is too generic and does not exploit task-specific characteristics. For
example, while generating descriptions from a table, a human would attend to
information at two levels: (i) the fields (macro level) and (ii) the values
within the field (micro level). Further, a human would continue attending to a
field for a few timesteps till all the information from that field has been
rendered and then never return back to this field (because there is nothing
left to say about it). To capture this behavior we use (i) a fused bifocal
attention mechanism which exploits and combines this micro and macro level
information and (ii) a gated orthogonalization mechanism which tries to ensure
that a field is remembered for a few time steps and then forgotten. We
experiment with a recently released dataset which contains fact tables about
people and their corresponding one line biographical descriptions in English.
In addition, we also introduce two similar datasets for French and German. Our
experiments show that the proposed model gives 21% relative improvement over a
recently proposed state of the art method and 10% relative improvement over
basic seq2seq models. The code and the datasets developed as a part of this
work are publicly available.

本文研究了从结构化表格中生成自然语言描述的方法，提出了利用联合的层级注意力机制和门控正交化机制，可以提高描述生成质量，并在公开数据集上获得了相对其他方法更好的结果。