Script event prediction aims to predict the subsequent event given the
context. This requires the capability to infer the correlations between events.
Recent works have attempted to improve event correlation reasoning by using
pretrained language models and incorporating external knowledge~(e.g.,
discourse relations). Though promising results have been achieved, some
challenges still remain. First, the pretrained language models adopted by
current works ignore event-level knowledge, resulting in an inability to
capture the correlations between events well. Second, modeling correlations
between events with discourse relations is limited because it can only capture
explicit correlations between events with discourse markers, and cannot capture
many implicit correlations. To this end, we propose a novel generative approach
for this task, in which a pretrained language model is fine-tuned with an
event-centric pretraining objective and predicts the next event within a
generative paradigm. Specifically, we first introduce a novel event-level blank
infilling strategy as the learning objective to inject event-level knowledge
into the pretrained language model, and then design a likelihood-based
contrastive loss for fine-tuning the generative model. Instead of using an
additional prediction layer, we perform prediction by using sequence
likelihoods generated by the generative model. Our approach models correlations
between events in a soft way without any external knowledge. The
likelihood-based prediction eliminates the need to use additional networks to
make predictions and is somewhat interpretable since it scores each word in the
event. Experimental results on the multi-choice narrative cloze~(MCNC) task
demonstrate that our approach achieves better results than other
state-of-the-art baselines. Our code will be available at
this https URL.

本文提出了一种基于事件中心预训练的生成式方法，其使用预先训练的语言模型 Fine-Tune 并通过对序列可能性的预测进行预测，从而软建模事件之间的关联。