We introduce Uncertain Natural Language Inference (UNLI), a refinement of
Natural Language Inference (NLI) that shifts away from categorical labels,
targeting instead the direct prediction of subjective probability assessments.
We demonstrate the feasibility of collecting annotations for UNLI by relabeling
a portion of the SNLI dataset under a probabilistic scale, where items even
with the same categorical label differ in how likely people judge them to be
true given a premise. We describe a direct scalar regression modeling approach,
and find that existing categorically labeled NLI data can be used in
pre-training. Our best models approach human performance, demonstrating models
may be capable of more subtle inferences than the categorical bin assignment
employed in current NLI tasks.

本文介绍了不确定的自然语言推断（UNLI），它是自然语言推断（NLI）的一种改进，通过预测主观概率评估来预测结果，利用概率评估重新标记了部分 SNLI 数据集，并使用现有的分类标签化的 NLI 数据进行预训练，最终模型展示了超越分类标签化的 NLI 任务的更微妙推断的人类表现水平。