Sequential modelling with self-attention has achieved cutting edge
performances in natural language processing. With advantages in model
flexibility, computation complexity and interpretability, self-attention is
gradually becoming a key component in event sequence models. However, like most
other sequence models, self-attention does not account for the time span
between events and thus captures sequential signals rather than temporal
patterns. Without relying on recurrent network structures, self-attention
recognizes event orderings via positional encoding. To bridge the gap between
modelling time-independent and time-dependent event sequence, we introduce a
functional feature map that embeds time span into high-dimensional spaces. By
constructing the associated translation-invariant time kernel function, we
reveal the functional forms of the feature map under classic functional
function analysis results, namely Bochner's Theorem and Mercer's Theorem. We
propose several models to learn the functional time representation and the
interactions with event representation. These methods are evaluated on
real-world datasets under various continuous-time event sequence prediction
tasks. The experiments reveal that the proposed methods compare favorably to
baseline models while also capturing useful time-event interactions.

使用自我关注的时间序列建模在自然语言处理中具有前沿技术，但是并不能捕捉到事件之间的时间跨度，我们通过使用高维函数特征映射来构建一个时间不变的核函数，提出了几种学习功能时间表示和与事件表示交互的模型，并在真实世界数据集上进行了实验，结果表明这些方法可以更好地捕捉时间事件交互。