We investigate contract element extraction. We show that LSTM-based encoders
perform better than dilated CNNs, Transformers, and BERT in this task. We also
find that domain-specific WORD2VEC embeddings outperform generic pre-trained
GLOVE embeddings. Morpho-syntactic features in the form of POS tag and token
shape embeddings, as well as context-aware ELMO embeddings do not improve
performance. Several of these observations contradict choices or findings of
previous work on contract element extraction and generic sequence labeling
tasks, indicating that contract element extraction requires careful
task-specific choices. Analyzing the results of (i) plain TRANSFORMER-based and
(ii) BERT-based models, we find that in the examined task, where the entities
are highly context-sensitive, the lack of recurrency in TRANSFORMERs greatly
affects their performance.

本文研究了合同元素提取，表明在这一任务中，基于 LSTM 的编码器比扩张卷积神经网络、Transformer 和 BERT 表现更好；领域特定的 WORD2VEC 嵌入优于通用预训练的 GLOVE 嵌入；形态句法特征和上下文感知的 ELMO 嵌入无法提高性能。这些观察结果与先前的合同元素提取和通用序列标注任务的研究结果相矛盾，表明合同元素提取需要仔细进行任务特定的选择。分析（i）基于标准 TRANSFORMER 的模型和（ii）基于 BERT 的模型的结果，我们发现在这一任务中，实体高度依赖于上下文，而标准 TRANSFORMER 中缺乏的循环结构显著影响了它们的性能。

神经合同要素提取再审视：来自芝麻街的信件

Neural Contract Element Extraction Revisited: Letters from Sesame Street

The recognition of pharmacological substances, compounds and proteins is an
essential preliminary work for the recognition of relations between chemicals
and other biomedically relevant units. In this paper, we describe an approach
to Task 1 of the PharmaCoNER Challenge, which involves the recognition of
mentions of chemicals and drugs in Spanish medical texts. We train a
state-of-the-art BiLSTM-CRF sequence tagger with stacked Pooled Contextualized
Embeddings, word and sub-word embeddings using the open-source framework FLAIR.
We present a new corpus composed of articles and papers from Spanish health
science journals, termed the Spanish Health Corpus, and use it to train
domain-specific embeddings which we incorporate in our model training. We
achieve a result of 89.76% F1-score using pre-trained embeddings and are able
to improve these results to 90.52% F1-score using specialized embeddings.

本研究基于 PharmaCoNER 挑战任务 1，使用 BiLSTM-CRF 序列标记器和基于 FLAIR 的深度学习方法，通过构建一个新的 Spanish Health Corpus 并使用领域专用嵌入，成功提高了化学药品和药物的识别精度。