Recently, neural language representation models pre-trained on large corpus
can capture rich co-occurrence information and be fine-tuned in downstream
tasks to improve the performance. As a result, they have achieved
state-of-the-art results in a large range of language tasks. However, there
exists other valuable semantic information such as similar, opposite, or other
possible meanings in external knowledge graphs (KGs). We argue that entities in
KGs could be used to enhance the correct semantic meaning of language
sentences. In this paper, we propose a new method CKG: Dynamic Representation
Based on \textbf{C}ontext and \textbf{K}nowledge \textbf{G}raph. On the one
side, CKG can extract rich semantic information of large corpus. On the other
side, it can make full use of inside information such as co-occurrence in large
corpus and outside information such as similar entities in KGs. We conduct
extensive experiments on a wide range of tasks, including QQP, MRPC, SST-5,
SQuAD, CoNLL 2003, and SNLI. The experiment results show that CKG achieves SOTA
89.2 on SQuAD compared with SAN (84.4), ELMo (85.8), and BERT$_{Base}$ (88.5).

本文提出了一种名为 CKG 的动态表示方法，利用外部知识图谱中的实体信息来提高语言句子的正确语义，实验表明，CKG 在多项任务中达到了最优结果。