An ultimate goal of artificial intelligence is to build computer systems that
can understand human languages. Understanding commonsense knowledge about the
world expressed in text is one of the foundational and challenging problems to
create such intelligent systems. As a step towards this goal, we present in
this paper ALMEn, an Autoencoding Language Model based Ensemble learning method
for commonsense validation and explanation. By ensembling several advanced
pre-trained language models including RoBERTa, DeBERTa, and ELECTRA with
Siamese neural networks, our method can distinguish natural language statements
that are against commonsense (validation subtask) and correctly identify the
reason for making against commonsense (explanation selection subtask).
Experimental results on the benchmark dataset of SemEval-2020 Task 4 show that
our method outperforms state-of-the-art models, reaching 97.9% and 95.4%
accuracies on the validation and explanation selection subtasks, respectively.

本文提出了一种基于自编码语言模型的集成学习方法，称为 ALMEn，用于常识验证和解释，在 SemEval-2020 任务 4 的基准数据集上表现良好，可区分违反常识的自然语言陈述并正确识别其原因，验证和解释选择子任务的准确性分别达到 97.9％和 95.4％，超过了现有的最先进模型。