Our physical world is constantly evolving over time, rendering challenges for
pre-trained language models to understand and reason over the temporal contexts
of texts. Existing work focuses on strengthening the direct association between
a piece of text and its time-stamp. However, the knowledge-time association is
usually insufficient for the downstream tasks that require reasoning over
temporal dependencies between knowledge. In this work, we make use of the
underlying nature of time, all temporally-scoped sentences are strung together
through a one-dimensional time axis, and suggest creating a graph structure
based on the relative placements of events along the time axis. Inspired by the
graph view, we propose RemeMo ($\underline{Re}$lative Ti$\underline{me}$
$\underline{Mo}$deling), which explicitly connects all temporally-scoped facts
by modeling the time relations between any two sentences. Experimental results
show that RemeMo outperforms the baseline T5 on multiple temporal question
answering datasets under various settings. Further analysis suggests that
RemeMo is especially good at modeling long-range complex temporal dependencies.
We release our code and pre-trained checkpoints at
$\href{this https URL}{\text{this url}}$.

本文提出了 RemeMo（相对时间建模）模型，通过建立基于时间轴上事件相对位置的图结构，明确地连接所有具有时间范围的事实，并模拟任意两个句子之间的时间关系，实验结果表明 RemeMo 在多个时间问答数据集上的性能优于基准模型 T5，特别擅长建模长距离复杂的时间依赖关系。