Pre-trained models (PTMs) have lead to great improvements in natural language
generation (NLG). However, it is still unclear how much commonsense knowledge
they possess. With the goal of evaluating commonsense knowledge of NLG models,
recent work has proposed the problem of generative commonsense reasoning, e.g.,
to compose a logical sentence given a set of unordered concepts. Existing
approaches to this problem hypothesize that PTMs lack sufficient parametric
knowledge for this task, which can be overcome by introducing external
knowledge or task-specific pre-training objectives. Different from this trend,
we argue that PTM's inherent ability for generative commonsense reasoning is
underestimated due to the order-agnostic property of its input. In particular,
we hypothesize that the order of the input concepts can affect the PTM's
ability to utilize its commonsense knowledge. To this end, we propose a
pre-ordering approach to elaborately manipulate the order of the given concepts
before generation. Experiments show that our approach can outperform the more
sophisticated models that have access to a lot of external data and resources.

本文研究了预训练模型的常识知识，提出使用预排序方法以改变输入概念的顺序，进而并非依靠外部知识和资源，有效提高模型的常识推理能力。