We introduce CDBERT, a new learning paradigm that enhances the semantics
understanding ability of the Chinese PLMs with dictionary knowledge and
structure of Chinese characters. We name the two core modules of CDBERT as
Shuowen and Jiezi, where Shuowen refers to the process of retrieving the most
appropriate meaning from Chinese dictionaries and Jiezi refers to the process
of enhancing characters' glyph representations with structure understanding. To
facilitate dictionary understanding, we propose three pre-training tasks, i.e.,
Masked Entry Modeling, Contrastive Learning for Synonym and Antonym, and
Example Learning. We evaluate our method on both modern Chinese understanding
benchmark CLUE and ancient Chinese benchmark CCLUE. Moreover, we propose a new
polysemy discrimination task PolyMRC based on the collected dictionary of
ancient Chinese. Our paradigm demonstrates consistent improvements on previous
Chinese PLMs across all tasks. Moreover, our approach yields significant
boosting on few-shot setting of ancient Chinese understanding.

CDBERT 是一种改进了语义理解的学习范例，它结合了语汇知识和汉字结构。该方法使用 Shuowen 和 Jiezi 两个核心模块，前者用于从汉字字典中获取最合适的意义，后者用于通过结构理解增强汉字的字形表示。通过在现代汉语理解基准 CLUE 和古代汉语基准 CCLUE 上评估，我们的方法表现出对以前的汉语 PLM 的一致提高。此外，我们在收集的古代汉字字典上提出了一种新的多义词鉴别任务 PolyMRC，并在古代汉语理解的少量数据情况下获得了显著的提升。