Sentence order prediction is the task of finding the correct order of
sentences in a randomly ordered document. Correctly ordering the sentences
requires an understanding of coherence with respect to the chronological
sequence of events described in the text. Document-level contextual
understanding and commonsense knowledge centered around these events are often
essential in uncovering this coherence and predicting the exact chronological
order. In this paper, we introduce STaCK -- a framework based on graph neural
networks and temporal commonsense knowledge to model global information and
predict the relative order of sentences. Our graph network accumulates temporal
evidence using knowledge of `past' and `future' and formulates sentence
ordering as a constrained edge classification problem. We report results on
five different datasets, and empirically show that the proposed method is
naturally suitable for order prediction. The implementation of this work is
publicly available at: this https URL

本篇论文介绍了一个名为 STaCK 的框架，基于图神经网络和时间常识知识对全局信息进行建模和预测句子的相对顺序，证明本文提出的方法非常适合进行序列预测问题。

STaCK：利用时间常识知识的句子排序

STaCK: Sentence Ordering with Temporal Commonsense Knowledge

As humans, we understand events in the visual world contextually, performing
multimodal reasoning across time to make inferences about the past, present,
and future. We introduce MERLOT, a model that learns multimodal script
knowledge by watching millions of YouTube videos with transcribed speech -- in
an entirely label-free, self-supervised manner. By pretraining with a mix of
both frame-level (spatial) and video-level (temporal) objectives, our model not
only learns to match images to temporally corresponding words, but also to
contextualize what is happening globally over time. As a result, MERLOT
exhibits strong out-of-the-box representations of temporal commonsense, and
achieves state-of-the-art performance on 12 different video QA datasets when
finetuned. It also transfers well to the world of static images, allowing
models to reason about the dynamic context behind visual scenes. On Visual
Commonsense Reasoning, MERLOT answers questions correctly with 80.6% accuracy,
outperforming state-of-the-art models of similar size by over 3%, even those
that make heavy use of auxiliary supervised data (like object bounding boxes).
Ablation analyses demonstrate the complementary importance of: 1) training on
videos versus static images; 2) scaling the magnitude and diversity of the
pretraining video corpus; and 3) using diverse objectives that encourage
full-stack multimodal reasoning, from the recognition to cognition level.

MERLOT 是一个模型，通过观看数百万个 YouTube 视频进行自我监督的无标签学习，学习多模态脚本知识，包括空间和时间和语境化的全局情况，从而在视觉场景中推理动态情境，形成出色的时间常识和现场表现，并在视觉常识推理方面表现出色。

MERLOT: 多模态神经脚本知识模型

MERLOT: Multimodal Neural Script Knowledge Models

Understanding time is crucial for understanding events expressed in natural
language. Because people rarely say the obvious, it is often necessary to have
commonsense knowledge about various temporal aspects of events, such as
duration, frequency, and temporal order. However, this important problem has so
far received limited attention. This paper systematically studies this temporal
commonsense problem. Specifically, we define five classes of temporal
commonsense, and use crowdsourcing to develop a new dataset, MCTACO, that
serves as a test set for this task. We find that the best current methods used
on MCTACO are still far behind human performance, by about 20%, and discuss
several directions for improvement. We hope that the new dataset and our study
here can foster more future research on this topic.

本文系统地研究了涉及持续时间、频率和时间顺序等事件的不同时间方面的常识问题并定义了五类时间常识。我们使用众包工具开发了一个新的测试集 MCTACO，研究表明，目前在 MCTACO 上表现最好的方法仍然比人类表现差 20%，该研究指出了未来改进的方向。