Thanks to the strong representation power of neural encoders, neural
chart-based parsers have achieved highly competitive performance by using local
features. Recently, it has been shown that non-local features in CRF structures
lead to improvements. In this paper, we investigate injecting non-local
features into the training process of a local span-based parser, by predicting
constituent n-gram non-local patterns and ensuring consistency between
non-local patterns and local constituents. Results show that our simple method
gives better results than the self-attentive parser on both PTB and CTB.
Besides, our method achieves state-of-the-art BERT-based performance on PTB
(95.92 F1) and strong performance on CTB (92.31 F1). Our parser also achieves
better or competitive performance in multilingual and zero-shot cross-domain
settings compared with the baseline.

本文研究将非局部特征注入本地基于跨度的解析器的训练过程中，通过预测组成成分 n-gram 非局部模式并确保非局部模式与本地成分的一致性，结果显示，我们的方法在 PTB 和 CTB 上均优于自我注意解析器，在 PTB 上达到 BERT 的最新性能（95.92 F1），并在 CTB 上获得强大的性能（92.31 F1）。与基线相比，在多语言和零 - shot 跨领域设置中，我们的解析器也实现了更好或具有竞争力的性能。

探究神经组成句法分析的非局部特征

Investigating Non-local Features for Neural Constituency Parsing

Word alignment is an important natural language processing task that
indicates the correspondence between natural languages. Recently, unsupervised
learning of log-linear models for word alignment has received considerable
attention as it combines the merits of generative and discriminative
approaches. However, a major challenge still remains: it is intractable to
calculate the expectations of non-local features that are critical for
capturing the divergence between natural languages. We propose a contrastive
approach that aims to differentiate observed training examples from noises. It
not only introduces prior knowledge to guide unsupervised learning but also
cancels out partition functions. Based on the observation that the probability
mass of log-linear models for word alignment is usually highly concentrated, we
propose to use top-n alignments to approximate the expectations with respect to
posterior distributions. This allows for efficient and accurate calculation of
expectations of non-local features. Experiments show that our approach achieves
significant improvements over state-of-the-art unsupervised word alignment
methods.

提出一种对抗学习的对比法用于无监督 log-linear 模型的词对齐任务，通过使用 top-n 对齐来近似计算非局部特征的期望值，实现了高效准确的无监督词对齐，并在实验中取得了显著的改进。