Commonsense generation aims to generate a realistic sentence describing a
daily scene under the given concepts, which is very challenging, since it
requires models to have relational reasoning and compositional generalization
capabilities. Previous work focuses on retrieving prototype sentences for the
provided concepts to assist generation. They first use a sparse retriever to
retrieve candidate sentences, then re-rank the candidates with a ranker.
However, the candidates returned by their ranker may not be the most relevant
sentences, since the ranker treats all candidates equally without considering
their relevance to the reference sentences of the given concepts. Another
problem is that re-ranking is very expensive, but only using retrievers will
seriously degrade the performance of their generation models. To solve these
problems, we propose the metric distillation rule to distill knowledge from the
metric (e.g., BLEU) to the ranker. We further transfer the critical knowledge
summarized by the distilled ranker to the retriever. In this way, the relevance
scores of candidate sentences predicted by the ranker and retriever will be
more consistent with their quality measured by the metric. Experimental results
on the CommonGen benchmark verify the effectiveness of our proposed method: (1)
Our generation model with the distilled ranker achieves a new state-of-the-art
result. (2) Our generation model with the distilled retriever even surpasses
the previous SOTA.

该研究提出了一种基于 Metric Distillation Rule 的常识句子生成方法，通过传递重要性知识来使检索预测结果更加一致并提高性能。

度量引导下的蒸馏：从度量器到排序器和检索器的知识蒸馏，用于生成式常识推理

Metric-guided Distillation: Distilling Knowledge from the Metric to Ranker and Retriever for Generative Commonsense Reasoning

Commonsense generation is a challenging task of generating a plausible
sentence describing an everyday scenario using provided concepts. Its
requirement of reasoning over commonsense knowledge and compositional
generalization ability even puzzles strong pre-trained language generation
models. We propose a novel framework using retrieval methods to enhance both
the pre-training and fine-tuning for commonsense generation. We retrieve
prototype sentence candidates by concept matching and use them as auxiliary
input. For fine-tuning, we further boost its performance with a trainable
sentence retriever. We demonstrate experimentally on the large-scale CommonGen
benchmark that our approach achieves new state-of-the-art results.

采用检索方法增强先前训练和微调，提出了一种新的框架用于增强常识生成，通过原型句子候选集，进一步使用可训练的句子检索器提高其性能，并在大规模 CommonGen 基准测试上实现了最新的最优结果。

检索增强型通识生成模型

Retrieval Enhanced Model for Commonsense Generation

Commonsense generation aims at generating plausible everyday scenario
description based on a set of provided concepts. Digging the relationship of
concepts from scratch is non-trivial, therefore, we retrieve prototypes from
external knowledge to assist the understanding of the scenario for better
description generation. We integrate two additional modules, namely position
indicator and scaling module, into the pretrained encoder-decoder model for
prototype modeling to enhance the knowledge injection procedure. We conduct
experiment on CommonGen benchmark, and experimental results show that our
method significantly improves the performance on all the metrics.

利用外部知识概念辅助 CommonSense 生成，通过原型建模和知识注入方法显著提高了性能。