In this paper, we focus on task-specific question answering (QA). To this
end, we introduce a method for generating exhaustive and high-quality training
data, which allows us to train compact (e.g., run on a mobile device),
task-specific QA models that are competitive against GPT variants. The key
technological enabler is a novel mechanism for automatic question-answer
generation from procedural text which can ingest large amounts of textual
instructions and produce exhaustive in-domain QA training data. While current
QA data generation methods can produce well-formed and varied data, their
non-exhaustive nature is sub-optimal for training a QA model. In contrast, we
leverage the highly structured aspect of procedural text and represent each
step and the overall flow of the procedure as graphs. We then condition on
graph nodes to automatically generate QA pairs in an exhaustive and
controllable manner. Comprehensive evaluations of our method show that: 1)
small models trained with our data achieve excellent performance on the target
QA task, even exceeding that of GPT3 and ChatGPT despite being several orders
of magnitude smaller. 2) semantic coverage is the key indicator for downstream
QA performance. Crucially, while large language models excel at syntactic
diversity, this does not necessarily result in improvements on the end QA
model. In contrast, the higher semantic coverage provided by our method is
critical for QA performance.

我们提出了一种使用自动生成的训练数据生成高性能任务特定问答模型的方法，该方法可以在手机等终端上运行且竞争力强，通过从过程性文本中自动生成详尽的领域内问答对进行训练，结构化的过程性文本是关键的技术支持。

面向程序化问答的图引导问题回答生成

Graph Guided Question Answer Generation for Procedural  Question-Answering

Recently, compressive text summarisation offers a balance between the
conciseness issue of extractive summarisation and the factual hallucination
issue of abstractive summarisation. However, most existing compressive
summarisation methods are supervised, relying on the expensive effort of
creating a new training dataset with corresponding compressive summaries. In
this paper, we propose an efficient and interpretable compressive summarisation
method that utilises unsupervised dual-agent reinforcement learning to optimise
a summary's semantic coverage and fluency by simulating human judgment on
summarisation quality. Our model consists of an extractor agent and a
compressor agent, and both agents have a multi-head attentional pointer-based
structure. The extractor agent first chooses salient sentences from a document,
and then the compressor agent compresses these extracted sentences by selecting
salient words to form a summary without using reference summaries to compute
the summary reward. To our best knowledge, this is the first work on
unsupervised compressive summarisation. Experimental results on three widely
used datasets (e.g., Newsroom, CNN/DM, and XSum) show that our model achieves
promising performance and a significant improvement on Newsroom in terms of the
ROUGE metric, as well as interpretability of semantic coverage of summarisation
results.

提出了一种利用无监督双代理强化学习来优化摘要语义覆盖和流畅度的高效且可解释的压缩摘要方法，该模型由提取器代理和压缩器代理组成，并且两个代理都具有多头注意力指针结构。实验结果表明，该模型在广泛使用的三个数据集上均取得了有希望的表现。

无监督双智能体强化学习实现高效可解释的压缩文本摘要

Efficient and Interpretable Compressive Text Summarisation with  Unsupervised Dual-Agent Reinforcement Learning

Data-to-text generation systems aim to generate text descriptions based on
input data (often represented in the tabular form). A typical system uses huge
training samples for learning the correspondence between tables and texts.
However, large training sets are expensive to obtain, limiting the
applicability of these approaches in real-world scenarios. In this work, we
focus on few-shot data-to-text generation. We observe that, while fine-tuned
pretrained language models may generate plausible sentences, they suffer from
the low semantic coverage problem in the few-shot setting. In other words,
important input slots tend to be missing in the generated text. To this end, we
propose a search-and-learning approach that leverages pretrained language
models but inserts the missing slots to improve the semantic coverage. We
further fine-tune our system based on the search results to smooth out the
search noise, yielding better-quality text and improving inference efficiency
to a large extent. Experiments show that our model achieves high performance on
E2E and WikiBio datasets. Especially, we cover 98.35% of input slots on E2E,
largely alleviating the low coverage problem.

该论文提出一种搜索和学习的方法来解决低语义覆盖率问题，进而提高 data-to-text 生成中预训练语言模型的效率，并在 E2E 和 WikiBio 数据集上实现了高性能。