Temporal Point Processes (TPPs), especially Hawkes Process are commonly used
for modeling asynchronous event sequences data such as financial transactions
and user behaviors in social networks. Due to the strong fitting ability of
neural networks, various neural Temporal Point Processes are proposed, among
which the Neural Hawkes Processes based on self-attention such as Transformer
Hawkes Process (THP) achieve distinct performance improvement. Although the THP
has gained increasing studies, it still suffers from the {sequence prediction
issue}, i.e., training on history sequences and inferencing about the future,
which is a prevalent paradigm in realistic sequence analysis tasks. What's
more, conventional THP and its variants simply adopt initial sinusoid embedding
in transformers, which shows performance sensitivity to temporal change or
noise in sequence data analysis by our empirical study. To deal with the
problems, we propose a new Rotary Position Embedding-based THP (RoTHP)
architecture in this paper. Notably, we show the translation invariance
property and {sequence prediction flexibility} of our RoTHP induced by the
{relative time embeddings} when coupled with Hawkes process theoretically.
Furthermore, we demonstrate empirically that our RoTHP can be better
generalized in sequence data scenarios with timestamp translations and in
sequence prediction tasks.

本文中，我们提出了一种基于旋转位置嵌入的 THP (RoTHP) 架构，通过相对时间嵌入与 Hawkes 过程的耦合，理论上展示了 RoTHP 的平移不变性和序列预测灵活性，同时在时间戳平移和序列预测任务中进行的实证研究表明 RoTHP 在序列数据场景中具有更好的泛化性。