Many natural language processing tasks, e.g., coreference resolution and
semantic role labeling, require selecting text spans and making decisions about
them.
A typical approach to such tasks is to score all possible spans and greedily
select spans for task-specific downstream processing.
This approach, however, does not incorporate any inductive bias about what
sort of spans ought to be selected, e.g., that selected spans tend to be
syntactic constituents.
In this paper, we propose a novel grammar-based structured span selection
model which learns to make use of the partial span-level annotation provided
for such problems.
Compared to previous approaches, our approach gets rid of the heuristic
greedy span selection scheme, allowing us to model the downstream task on an
optimal set of spans. We evaluate our model on two popular span prediction
tasks: coreference resolution and semantic role labeling.
We show empirical improvements on both.

本研究提出了一种基于语法的结构化跨度选择模型，旨在改进自然语言处理中的跨度选择问题，经测试该模型在两种跨度预测任务中均有显著提升。

结构化跨度选择器

A Structured Span Selector

We present SpanBERT, a pre-training method that is designed to better
represent and predict spans of text. Our approach extends BERT by (1) masking
contiguous random spans, rather than random tokens, and (2) training the span
boundary representations to predict the entire content of the masked span,
without relying on the individual token representations within it. SpanBERT
consistently outperforms BERT and our better-tuned baselines, with substantial
gains on span selection tasks such as question answering and coreference
resolution. In particular, with the same training data and model size as
BERT-large, our single model obtains 94.6% and 88.7% F1 on SQuAD 1.1 and 2.0,
respectively. We also achieve a new state of the art on the OntoNotes
coreference resolution task (79.6\% F1), strong performance on the TACRED
relation extraction benchmark, and even show gains on GLUE.

本论文提出了 SpanBERT 预训练方法，扩展了 BERT 的掩蔽方式和训练方式，使其在跨度选择和指代消解等任务中表现出比 BERT 和基线更优秀的性能，包括在 SQuAD、OntoNotes 和 TACRED 等各种基准测试上都取得了更好的结果。