In many natural language processing (NLP) tasks the same input (e.g. source
sentence) can have multiple possible outputs (e.g. translations). To analyze
how this ambiguity (also known as intrinsic uncertainty) shapes the
distribution learned by neural sequence models we measure sentence-level
uncertainty by computing the degree of overlap between references in
multi-reference test sets from two different NLP tasks: machine translation
(MT) and grammatical error correction (GEC). At both the sentence- and the
task-level, intrinsic uncertainty has major implications for various aspects of
search such as the inductive biases in beam search and the complexity of exact
search. In particular, we show that well-known pathologies such as a high
number of beam search errors, the inadequacy of the mode, and the drop in
system performance with large beam sizes apply to tasks with high level of
ambiguity such as MT but not to less uncertain tasks such as GEC. Furthermore,
we propose a novel exact $n$-best search algorithm for neural sequence models,
and show that intrinsic uncertainty affects model uncertainty as the model
tends to overly spread out the probability mass for uncertain tasks and
sentences.

通过测量自然语言处理中 intristic uncertainty 的程度，我们发现它会对搜索过程的归纳偏差和精确搜索的复杂度等方面产生重大影响，并提出了一种新的精确搜索算法用于神经序列模型，发现 intristic uncertainty 会影响模型的不确定性，从而导致概率质量过度分散。