Event sequence models have been found to be highly effective in the analysis
and prediction of events. Building such models requires availability of
abundant high-quality event sequence data. In certain applications, however,
clean structured event sequences are not available, and automated sequence
extraction results in data that is too noisy and incomplete. In this work, we
explore the use of Large Language Models (LLMs) to generate event sequences
that can effectively be used for probabilistic event model construction. This
can be viewed as a mechanism of distilling event sequence knowledge from LLMs.
Our approach relies on a Knowledge Graph (KG) of event concepts with partial
causal relations to guide the generative language model for causal event
sequence generation. We show that our approach can generate high-quality event
sequences, filling a knowledge gap in the input KG. Furthermore, we explore how
the generated sequences can be leveraged to discover useful and more complex
structured knowledge from pattern mining and probabilistic event models. We
release our sequence generation code and evaluation framework, as well as
corpus of event sequence data.

利用大型语言模型生成事件序列，填补知识图中的知识空白，并通过模式挖掘和概率事件模型发现有用的和更复杂的结构化知识。

从大型语言模型中提取事件序列知识

Distilling Event Sequence Knowledge From Large Language Models

Sequential modelling with self-attention has achieved cutting edge
performances in natural language processing. With advantages in model
flexibility, computation complexity and interpretability, self-attention is
gradually becoming a key component in event sequence models. However, like most
other sequence models, self-attention does not account for the time span
between events and thus captures sequential signals rather than temporal
patterns. Without relying on recurrent network structures, self-attention
recognizes event orderings via positional encoding. To bridge the gap between
modelling time-independent and time-dependent event sequence, we introduce a
functional feature map that embeds time span into high-dimensional spaces. By
constructing the associated translation-invariant time kernel function, we
reveal the functional forms of the feature map under classic functional
function analysis results, namely Bochner's Theorem and Mercer's Theorem. We
propose several models to learn the functional time representation and the
interactions with event representation. These methods are evaluated on
real-world datasets under various continuous-time event sequence prediction
tasks. The experiments reveal that the proposed methods compare favorably to
baseline models while also capturing useful time-event interactions.

使用自我关注的时间序列建模在自然语言处理中具有前沿技术，但是并不能捕捉到事件之间的时间跨度，我们通过使用高维函数特征映射来构建一个时间不变的核函数，提出了几种学习功能时间表示和与事件表示交互的模型，并在真实世界数据集上进行了实验，结果表明这些方法可以更好地捕捉时间事件交互。