Chinese sequence labeling tasks are heavily reliant on accurate word boundary
demarcation. Although current pre-trained language models (PLMs) have achieved
substantial gains on these tasks, they rarely explicitly incorporate boundary
information into the modeling process. An exception to this is BABERT, which
incorporates unsupervised statistical boundary information into Chinese BERT's
pre-training objectives. Building upon this approach, we input supervised
high-quality boundary information to enhance BABERT's learning, developing a
semi-supervised boundary-aware PLM. To assess PLMs' ability to encode
boundaries, we introduce a novel ``Boundary Information Metric'' that is both
simple and effective. This metric allows comparison of different PLMs without
task-specific fine-tuning. Experimental results on Chinese sequence labeling
datasets demonstrate that the improved BABERT variant outperforms the vanilla
version, not only on these tasks but also more broadly across a range of
Chinese natural language understanding tasks. Additionally, our proposed metric
offers a convenient and accurate means of evaluating PLMs' boundary awareness.

通过将监督边界信息用于 BABERT，我们构建了一种半监督的边界感知 PLM，实验结果表明我们改进的 BABERT 在中文序列标注和自然语言理解任务中表现出更好的性能，并且我们提出的度量方法具有方便和准确的评估 PLM 边界感知能力的特点。

基于半监督边界感知语言模型预训练的中文序列标注

Chinese Sequence Labeling with Semi-Supervised Boundary-Aware Language  Model Pre-training

Recently, it is quite common to integrate Chinese sequence labeling results
to enhance syntactic and semantic parsing. However, little attention has been
paid to the utility of hierarchy and structure information encoded in syntactic
and semantic features for Chinese sequence labeling tasks. In this paper, we
propose a novel framework to encode syntactic structure features and semantic
information for Chinese sequence labeling tasks with graph convolutional
networks (GCN). Experiments on five benchmark datasets, including Chinese word
segmentation and part-of-speech tagging, demonstrate that our model can
effectively improve the performance of Chinese labeling tasks.

本文提出了一种新型的桥式结构特征和语义信息的汉语序列标注任务编码框架，并使用图卷积网络对其进行实现，实验结果表明，该模型可以有效提高汉语标注任务的性能。

应用 GCN 结合深度句法和语义知识进行中文序列标注

Incorporating Deep Syntactic and Semantic Knowledge for Chinese Sequence  Labeling with GCN

Boundary information is critical for various Chinese language processing
tasks, such as word segmentation, part-of-speech tagging, and named entity
recognition. Previous studies usually resorted to the use of a high-quality
external lexicon, where lexicon items can offer explicit boundary information.
However, to ensure the quality of the lexicon, great human effort is always
necessary, which has been generally ignored. In this work, we suggest
unsupervised statistical boundary information instead, and propose an
architecture to encode the information directly into pre-trained language
models, resulting in Boundary-Aware BERT (BABERT). We apply BABERT for feature
induction of Chinese sequence labeling tasks. Experimental results on ten
benchmarks of Chinese sequence labeling demonstrate that BABERT can provide
consistent improvements on all datasets. In addition, our method can complement
previous supervised lexicon exploration, where further improvements can be
achieved when integrated with external lexicon information.

该论文提出了一种在预训练语言模型中直接编码无监督统计边界信息的体系结构 Boundary-Aware BERT (BABERT)，并将其应用于中文序列标注任务的特征诱导，实验结果表明 BABERT 在十个中文序列标注基准上均可提供一致的改善，同时，该方法可以补充先前监督词典探索的不足，实现与外部词典信息的集成以进一步提高性能。