Temporal Point Processes (TPPs), especially Hawkes Process are commonly used
for modeling asynchronous event sequences data such as financial transactions
and user behaviors in social networks. Due to the strong fitting ability of
neural networks, various neural Temporal Point Processes are proposed, among
which the Neural Hawkes Processes based on self-attention such as Transformer
Hawkes Process (THP) achieve distinct performance improvement. Although the THP
has gained increasing studies, it still suffers from the {sequence prediction
issue}, i.e., training on history sequences and inferencing about the future,
which is a prevalent paradigm in realistic sequence analysis tasks. What's
more, conventional THP and its variants simply adopt initial sinusoid embedding
in transformers, which shows performance sensitivity to temporal change or
noise in sequence data analysis by our empirical study. To deal with the
problems, we propose a new Rotary Position Embedding-based THP (RoTHP)
architecture in this paper. Notably, we show the translation invariance
property and {sequence prediction flexibility} of our RoTHP induced by the
{relative time embeddings} when coupled with Hawkes process theoretically.
Furthermore, we demonstrate empirically that our RoTHP can be better
generalized in sequence data scenarios with timestamp translations and in
sequence prediction tasks.

本文中，我们提出了一种基于旋转位置嵌入的 THP (RoTHP) 架构，通过相对时间嵌入与 Hawkes 过程的耦合，理论上展示了 RoTHP 的平移不变性和序列预测灵活性，同时在时间戳平移和序列预测任务中进行的实证研究表明 RoTHP 在序列数据场景中具有更好的泛化性。

RoTHP：基于旋转位置嵌入的变压器霍克斯过程

RoTHP: Rotary Position Embedding-based Transformer Hawkes Process

Transformer Hawkes process models have shown to be successful in modeling
event sequence data. However, most of the existing training methods rely on
maximizing the likelihood of event sequences, which involves calculating some
intractable integral. Moreover, the existing methods fail to provide
uncertainty quantification for model predictions, e.g., confidence intervals
for the predicted event's arrival time. To address these issues, we propose
SMURF-THP, a score-based method for learning Transformer Hawkes process and
quantifying prediction uncertainty. Specifically, SMURF-THP learns the score
function of events' arrival time based on a score-matching objective that
avoids the intractable computation. With such a learned score function, we can
sample arrival time of events from the predictive distribution. This naturally
allows for the quantification of uncertainty by computing confidence intervals
over the generated samples. We conduct extensive experiments in both event type
prediction and uncertainty quantification of arrival time. In all the
experiments, SMURF-THP outperforms existing likelihood-based methods in
confidence calibration while exhibiting comparable prediction accuracy.

我们提出了 SMURF-THP，一种基于得分的学习方法，用于学习 Transformer Hawkes 过程并量化预测不确定性。通过学习到的得分函数，我们可以从预测分布中采样事件的到达时间，从而计算生成样本的置信区间，实现了对不确定性的量化。在事件类型预测和到达时间不确定性量化的实验中，SMURF-THP 表现优于现有的基于似然的方法，同时展示了可比较的预测准确性。

SMURF-THP：基于得分匹配的变压器霍克进程不确定性量化

SMURF-THP: Score Matching-based UnceRtainty quantiFication for  Transformer Hawkes Process

Modern data acquisition routinely produce massive amounts of event sequence
data in various domains, such as social media, healthcare, and financial
markets. These data often exhibit complicated short-term and long-term temporal
dependencies. However, most of the existing recurrent neural network based
point process models fail to capture such dependencies, and yield unreliable
prediction performance. To address this issue, we propose a Transformer Hawkes
Process (THP) model, which leverages the self-attention mechanism to capture
long-term dependencies and meanwhile enjoys computational efficiency. Numerical
experiments on various datasets show that THP outperforms existing models in
terms of both likelihood and event prediction accuracy by a notable margin.
Moreover, THP is quite general and can incorporate additional structural
knowledge. We provide a concrete example, where THP achieves improved
prediction performance for learning multiple point processes when incorporating
their relational information.

提出了一种基于 Transformer 和 Hawkes 过程的模型（THP），能够更好地捕捉事件序列数据中的短期和长期时间依赖关系，并能够通过集成结构性知识来改善预测表现。