The rapid growth of research publications has placed great demands on digital
libraries (DL) for advanced information management technologies. To cater to
these demands, techniques relying on knowledge-graph structures are being
advocated. In such graph-based pipelines, inferring semantic relations between
related scientific concepts is a crucial step. Recently, BERT-based pre-trained
models have been popularly explored for automatic relation classification.
Despite significant progress, most of them were evaluated in different
scenarios, which limits their comparability. Furthermore, existing methods are
primarily evaluated on clean texts, which ignores the digitization context of
early scholarly publications in terms of machine scanning and optical character
recognition (OCR). In such cases, the texts may contain OCR noise, in turn
creating uncertainty about existing classifiers' performances. To address these
limitations, we started by creating OCR-noisy texts based on three clean
corpora. Given these parallel corpora, we conducted a thorough empirical
evaluation of eight Bert-based classification models by focusing on three
factors: (1) Bert variants; (2) classification strategies; and, (3) OCR noise
impacts. Experiments on clean data show that the domain-specific pre-trained
Bert is the best variant to identify scientific relations. The strategy of
predicting a single relation each time outperforms the one simultaneously
identifying multiple relations in general. The optimal classifier's performance
can decline by around 10% to 20% in F-score on the noisy corpora. Insights
discussed in this study can help DL stakeholders select techniques for building
optimal knowledge-graph-based systems.

通过对三个语料库进行互相匹配，开展了 BERT 模型的实证研究。实验结果显示，与其他变量相比，该领域特定的预训练 BERT 变体最适合于关系检测任务，并且单次预测一个关系的策略通常比同时识别多个关系的策略表现更好，同时研究的结论还可以帮助数字图书馆利益相关者选择最佳的知识图谱辅助技术。