Neural network has become the dominant method for Chinese word segmentation.
Most existing models cast the task as sequence labeling, using BiLSTM-CRF for
representing the input and making output predictions. Recently, attention-based
sequence models have emerged as a highly competitive alternative to LSTMs,
which allow better running speed by parallelization of computation. We
investigate self attention network for Chinese word segmentation, making
comparisons between BiLSTM-CRF models. In addition, the influence of
contextualized character embeddings is investigated using BERT, and a method is
proposed for integrating word information into SAN segmentation. Results show
that SAN gives highly competitive results compared with BiLSTMs, with BERT and
word information further improving segmentation for in-domain and cross-domain
segmentation. Our final models give the best results for 6 heterogenous domain
benchmarks.

研究了自注意网络和 BERT 对中文分词任务的影响，提出了将词信息集成到自注意网络中的方法，并展示了该方法在 6 个领域基准测试中的最佳结果。

自我关注网络在中文分词中的应用研究

Investigating Self-Attention Network for Chinese Word Segmentation

Encoder-decoder based Sequence to Sequence learning (S2S) has made remarkable
progress in recent years. Different network architectures have been used in the
encoder/decoder. Among them, Convolutional Neural Networks (CNN) and Self
Attention Networks (SAN) are the prominent ones. The two architectures achieve
similar performances but use very different ways to encode and decode context:
CNN use convolutional layers to focus on the local connectivity of the
sequence, while SAN uses self-attention layers to focus on global semantics. In
this work we propose Double Path Networks for Sequence to Sequence learning
(DPN-S2S), which leverage the advantages of both models by using double path
information fusion. During the encoding step, we develop a double path
architecture to maintain the information coming from different paths with
convolutional layers and self-attention layers separately. To effectively use
the encoded context, we develop a cross attention module with gating and use it
to automatically pick up the information needed during the decoding step. By
deeply integrating the two paths with cross attention, both types of
information are combined and well exploited. Experiments show that our proposed
method can significantly improve the performance of sequence to sequence
learning over state-of-the-art systems.

本文提出了一种名为双路径网络的编码器 - 解码器模型，通过双重信息融合实现了卷积神经网络和自注意力网络各自的优势，利用交叉注意力模块进行编码解码。实验证明，该方法能够显著提高序列到序列学习的性能。