Incorporating lexical knowledge into deep learning models has been proved to be very effective for sequence labeling tasks. However, previous works commonly have difficulty dealing with large-scale dynamic lexicons which often cause excessive matching noise and problems of frequent updates. In this paper, we propose DyLex, a plug-in lexicon incorporation approach for BERT based sequence labeling tasks. Instead of leveraging embeddings of words in the lexicon as in conventional methods, we adopt word-agnostic tag embeddings to avoid re-training the representation while updating the lexicon. Moreover, we employ an effective supervised lexical knowledge denoising method to smooth out matching noise. Finally, we introduce a col-wise attention based knowledge fusion mechanism to guarantee the pluggability of the proposed framework. Experiments on ten datasets of three tasks show that the proposed framework achieves new SOTA, even with very large scale lexicons.

本文提出了DyLex：一种适用于基于BERT的序列标注任务的插件词库融合方法，避免了大规模动态词库的噪声和更新问题，采用了无关单词的标签嵌入和监督词库去噪方法，并引入基于列的注意力机制来保证该框架的可插入性。实验结果表明，该框架即使与非常大的词库一起使用也实现了最新的SOTA效果。

将动态词典加入BERT用于序列标记