BERT-based models have shown a remarkable ability in the Chinese Spelling
Check (CSC) task recently. However, traditional BERT-based methods still suffer
from two limitations. First, although previous works have identified that
explicit prior knowledge like Part-Of-Speech (POS) tagging can benefit in the
CSC task, they neglected the fact that spelling errors inherent in CSC data can
lead to incorrect tags and therefore mislead models. Additionally, they ignored
the correlation between the implicit hierarchical information encoded by BERT's
intermediate layers and different linguistic phenomena. This results in
sub-optimal accuracy. To alleviate the above two issues, we design a
heterogeneous knowledge-infused framework to strengthen BERT-based CSC models.
To incorporate explicit POS knowledge, we utilize an auxiliary task strategy
driven by Gaussian mixture model. Meanwhile, to incorporate implicit
hierarchical linguistic knowledge within the encoder, we propose a novel form
of n-gram-based layerwise self-attention to generate a multilayer
representation. Experimental results show that our proposed framework yields a
stable performance boost over four strong baseline models and outperforms the
previous state-of-the-art methods on two datasets.

基于 BERT 的模型在中文拼写检查任务方面表现出了显著的能力，然而传统的基于 BERT 的方法仍存在两个局限性：第一，虽然之前的研究发现明确的先验知识如词性标注对于拼写检查任务有益，但他们忽略了拼写错误会导致错误标签从而误导模型的事实；此外，他们忽视了 BERT 中间层所编码的隐含分层信息与不同的语言现象之间的相关性，从而导致了次优的准确性。我们提出了一种异构知识注入框架来减轻上述两个问题。为了整合明确的词性知识，我们采用了由高斯混合模型驱动的辅助任务策略。同时，为了整合编码器中的隐含的分层语言知识，我们提出了一种新颖的基于 n-gram 的逐层自注意力形式来生成多层表示。实验结果表明，我们提出的框架在四个强基准模型上都有稳定的性能提升，并且在两个数据集上优于先前的最先进方法。