Recently, large-scale pre-trained language models have demonstrated
impressive performance on several commonsense-reasoning benchmark datasets.
However, building machines with commonsense to compose realistically plausible
sentences remains challenging. In this paper, we present a constrained text
generation task, CommonGen associated with a benchmark dataset, to explicitly
test machines for the ability of generative commonsense reasoning. Given a set
of common concepts (e.g., {dog, frisbee, catch, throw}); the task is to
generate a coherent sentence describing an everyday scenario using these
concepts (e.g., "a man throws a frisbee and his dog catches it").
The CommonGen task is challenging because it inherently requires 1)
relational reasoning with background commonsense knowledge, and 2)
compositional generalization ability to work on unseen concept combinations.
Our dataset, constructed through a combination of crowdsourced and existing
caption corpora, consists of 79k commonsense descriptions over 35k unique
concept-sets. Experiments show that there is a large gap between
state-of-the-art text generation models (e.g., T5) and human performance.
Furthermore, we demonstrate that the learned generative commonsense reasoning
capability can be transferred to improve downstream tasks such as CommonsenseQA
by generating additional context.

本文介绍了一个基于 CommonGen 任务的数据集，用于评估预训练语言模型的生成常识推理能力，任务的难点在于需要进行关系推理并具有通用性，实验结果表明目前最先进的文本生成模型与人类表现存在很大差距，且生成的常识推理能力可以用于提高 Downstream 任务的性能。