The rapid advancements in large language models (LLMs) have ignited interest
in the temporal knowledge graph (tKG) domain, where conventional carefully
designed embedding-based and rule-based models dominate. The question remains
open of whether pre-trained LLMs can understand structured temporal relational
data and replace them as the foundation model for temporal relational
forecasting. Therefore, we bring temporal knowledge forecasting into the
generative setting. However, challenges occur in the huge chasms between
complex temporal graph data structure and sequential natural expressions LLMs
can handle, and between the enormous data sizes of tKGs and heavy computation
costs of finetuning LLMs. To address these challenges, we propose a novel
retrieval augmented generation framework that performs generative forecasting
on tKGs named GenTKG, which combines a temporal logical rule-based retrieval
strategy and lightweight parameter-efficient instruction tuning. Extensive
experiments have shown that GenTKG outperforms conventional methods of temporal
relational forecasting under low computation resources. GenTKG also highlights
remarkable transferability with exceeding performance on unseen datasets
without re-training. Our work reveals the huge potential of LLMs in the tKG
domain and opens a new frontier for generative forecasting on tKGs.

我们提出了一个新颖的检索增强生成框架 GenTKG，结合了时间逻辑规则检索策略和轻量级参数效率指导，用于在时间知识图上进行生成性预测，证实了大语言模型在时间知识图领域具有巨大潜力，并开辟了时间知识图生成性预测的新领域。

GenTKG：时序知识图谱上的生成预测

GenTKG: Generative Forecasting on Temporal Knowledge Graph

Long-range time series forecasting is usually based on one of two existing
forecasting strategies: Direct Forecasting and Iterative Forecasting, where the
former provides low bias, high variance forecasts and the latter leads to low
variance, high bias forecasts. In this paper, we propose a new forecasting
strategy called Generative Forecasting (GenF), which generates synthetic data
for the next few time steps and then makes long-range forecasts based on
generated and observed data. We theoretically prove that GenF is able to better
balance the forecasting variance and bias, leading to a much smaller
forecasting error. We implement GenF via three components: (i) a novel
conditional Wasserstein Generative Adversarial Network (GAN) based generator
for synthetic time series data generation, called CWGAN-TS. (ii) a transformer
based predictor, which makes long-range predictions using both generated and
observed data. (iii) an information theoretic clustering algorithm to improve
the training of both the CWGAN-TS and the transformer based predictor. The
experimental results on five public datasets demonstrate that GenF
significantly outperforms a diverse range of state-of-the-art benchmarks and
classical approaches. Specifically, we find a 5% - 11% improvement in
predictive performance (mean absolute error) while having a 15% - 50% reduction
in parameters compared to the benchmarks. Lastly, we conduct an ablation study
to further explore and demonstrate the effectiveness of the components
comprising GenF.

本文提出一种基于生成对抗网络及变换器的长期时间序列预测方法 (GenF), 证明其在预测方差和偏差上更加平衡，实验数据显示该方法在五个公共数据集上表现显著优于其他方法。