Neural language representation models such as BERT, pre-trained on
large-scale unstructured corpora lack explicit grounding to real-world
commonsense knowledge and are often unable to remember facts required for
reasoning and inference. Natural Language Inference (NLI) is a challenging
reasoning task that relies on common human understanding of language and
real-world commonsense knowledge. We introduce a new model for NLI called
External Knowledge Enhanced BERT (ExBERT), to enrich the contextual
representation with real-world commonsense knowledge from external knowledge
sources and enhance BERT's language understanding and reasoning capabilities.
ExBERT takes full advantage of contextual word representations obtained from
BERT and employs them to retrieve relevant external knowledge from knowledge
graphs and to encode the retrieved external knowledge. Our model adaptively
incorporates the external knowledge context required for reasoning over the
inputs. Extensive experiments on the challenging SciTail and SNLI benchmarks
demonstrate the effectiveness of ExBERT: in comparison to the previous
state-of-the-art, we obtain an accuracy of 95.9% on SciTail and 91.5% on SNLI.

本文提出了一种新的 NLI 模型 ——ExBERT，它可以利用外部知识源对 BERT 的上下文表示进行丰富，从而提升 BERT 的语言理解和推理能力，并在挑战性的 SciTail 和 SNLI 基准测试中实现了优异的表现。

ExBERT: 外部知识增强的自然语言推断 BERT 模型

ExBERT: An External Knowledge Enhanced BERT for Natural Language  Inference

Neural language representation models such as BERT pre-trained on large-scale
corpora can well capture rich semantic patterns from plain text, and be
fine-tuned to consistently improve the performance of various NLP tasks.
However, the existing pre-trained language models rarely consider incorporating
knowledge graphs (KGs), which can provide rich structured knowledge facts for
better language understanding. We argue that informative entities in KGs can
enhance language representation with external knowledge. In this paper, we
utilize both large-scale textual corpora and KGs to train an enhanced language
representation model (ERNIE), which can take full advantage of lexical,
syntactic, and knowledge information simultaneously. The experimental results
have demonstrated that ERNIE achieves significant improvements on various
knowledge-driven tasks, and meanwhile is comparable with the state-of-the-art
model BERT on other common NLP tasks. The source code of this paper can be
obtained from this https URL

本文提出了 ERNIE，使用大规模的文本语料库和知识图谱（KGs）同时训练，利用 KGs 中的外部知识来增强语言表征，实验证明在不同知识驱动任务上 ERNIE 显著提高性能，同时在其他常见 NLP 任务上与 BERT 的性能相当。