Commonsense question answering (CQA) aims to test if models can answer
questions regarding commonsense knowledge that everyone knows. Prior works that
incorporate external knowledge bases have shown promising results, but
knowledge bases are expensive to construct and are often limited to a fixed set
of relations. In this paper, we instead focus on better utilizing the
\textit{implicit knowledge} stored in pre-trained language models. While
researchers have found that the knowledge embedded in pre-trained language
models can be extracted by having them fill in the blanks of carefully designed
prompts for relation extraction and text classification, it remains unclear if
we can adopt this paradigm in CQA where the inputs and outputs take much more
flexible forms. To this end, we investigate four translation methods that can
translate natural questions into cloze-style sentences to better solicit
commonsense knowledge from language models, including a syntactic-based model,
an unsupervised neural model, and two supervised neural models. In addition, to
combine the different translation methods, we propose to encourage consistency
among model predictions on different translated questions with unlabeled data.
We demonstrate the effectiveness of our methods on three CQA datasets in
zero-shot settings. We show that our methods are complementary to a knowledge
base improved model, and combining them can lead to state-of-the-art zero-shot
performance. Analyses also reveal distinct characteristics of the different
cloze translation methods and provide insights on why combining them can lead
to great improvements.

本研究针对常识问题回答（Commonsense question answering）的挑战，提出了利用预训练语言模型中的 “隐含知识”，通过四种翻译方法，探索如何更好地调动语言模型中的常识知识。研究通过在零样本情况下对三个 CQA 数据集进行实验，证明了该方法对增强模型的性能十分有效，并且结合外部知识库进一步提高了模型性能。同时，文章还分析了各种填空翻译方法的不同特点和优缺点。

用填空翻译和一致性优化实现零样本通识常识问答

Zero-shot Commonsense Question Answering with Cloze Translation and Consistency Optimization

In this paper, we focus on a new practical task, document-scale text content
manipulation, which is the opposite of text style transfer and aims to preserve
text styles while altering the content. In detail, the input is a set of
structured records and a reference text for describing another recordset. The
output is a summary that accurately describes the partial content in the source
recordset with the same writing style of the reference. The task is
unsupervised due to lack of parallel data, and is challenging to select
suitable records and style words from bi-aspect inputs respectively and
generate a high-fidelity long document. To tackle those problems, we first
build a dataset based on a basketball game report corpus as our testbed, and
present an unsupervised neural model with interactive attention mechanism,
which is used for learning the semantic relationship between records and
reference texts to achieve better content transfer and better style
preservation. In addition, we also explore the effectiveness of the
back-translation in our task for constructing some pseudo-training pairs.
Empirical results show superiority of our approaches over competitive methods,
and the models also yield a new state-of-the-art result on a sentence-level
dataset.

本文提出了一个新的实际任务，文档级文本内容操作，它旨在在更改内容的同时保留文本样式，并提出一种基于交互式注意机制的无监督神经模型，探索了在该任务中构建伪训练对的后向翻译方法的有效性。实证结果表明，该方法优于竞争方法，并且模型还在句子级数据集上产生了新的最优结果