A considerable number of texts encountered daily are somehow connected with
each other. For example, Wikipedia articles refer to other articles via
hyperlinks, scientific papers relate to others via citations or (co)authors,
while tweets relate via users that follow each other or reshare content. Hence,
a graph-like structure can represent existing connections and be seen as
capturing the "context" of the texts. The question thus arises if extracting
and integrating such context information into a language model might help
facilitate a better automated understanding of the text. In this study, we
experimentally demonstrate that incorporating graph-based contextualization
into BERT model enhances its performance on an example of a classification
task. Specifically, on Pubmed dataset, we observed a reduction in error from
8.51% to 7.96%, while increasing the number of parameters just by 1.6%.
Our source code: this https URL

本研究实验证明在 BERT 模型中加入基于图形的上下文信息会提高模型性能，并在 Pubmed 数据集上取得了 8.51％到 7.96% 的错误率减少，同时仅增加 1.6％的参数。