Human activities generate various event sequences such as taxi trip records,
bike-sharing pick-ups, crime occurrence, and infectious disease transmission.
The point process is widely used in many applications to predict such events
related to human activities. However, point processes present two problems in
predicting events related to human activities. First, recent high-performance
point process models require the input of sufficient numbers of events
collected over a long period (i.e., long sequences) for training, which are
often unavailable in realistic situations. Second, the long-term predictions
required in real-world applications are difficult. To tackle these problems, we
propose a novel meta-learning approach for periodicity-aware prediction of
future events given short sequences. The proposed method first embeds short
sequences into hidden representations (i.e., task representations) via
recurrent neural networks for creating predictions from short sequences. It
then models the intensity of the point process by monotonic neural networks
(MNNs), with the input being the task representations. We transfer the prior
knowledge learned from related tasks and can improve event prediction given
short sequences of target tasks. We design the MNNs to explicitly take temporal
periodic patterns into account, contributing to improved long-term prediction
performance. Experiments on multiple real-world datasets demonstrate that the
proposed method has higher prediction performance than existing alternatives.

提出了一种用于周期性感知预测短序列未来事件的元学习方法，通过点过程建模和时间周期模式考虑来提高事件预测性能。

基于神经网络的时间点过程的元学习

Meta-Learning for Neural Network-based Temporal Point Processes

Sequential recommendation can capture user chronological preferences from
their historical behaviors, yet the learning of short sequences (cold-start
problem) in many benchmark datasets is still an open challenge. Recently, data
augmentation with pseudo-prior items generated by Transformers has drawn
considerable attention. These methods can generate pseudo-prior items
sequentially in reverse chronological order to extend the original sequences.
Nevertheless, the performance may still dramatically degrade in very short
sequences; most notably, the generation of pseudo-prior items does not take
into account the forward direction (from the past to the future), and so the
underlying temporal correlations are not preserved in terms of conditional
probabilities. Motivated by this, we propose a Bidirectional Chronological
Augmentation of Transformer (BiCAT) that uses a forward learning constraint in
the reverse generative process to capture contextual information more
effectively. Then, self-knowledge distillation is adopted in augmented and
original sequences to bridge the gap between data augmentation and model
representation, which enhances the robustness of sequence encoder. Moreover, an
informative positive and negative sampling strategy is proposed to accelerate
optimization and prevent overfitting. Extensive experiments on two popular
real-world datasets demonstrate the efficacy of our method on very short
sequences (L < 3) and long sequences (20 < L < 50) as well, our approach
outperforms state-of-the-art models by an average of 35.04% and 8.76%
respectively, in terms of Recall@5. Source code is available at
this https URL

研究通过双向时间排序转换数据增强和自主知识蒸馏提高顺序推荐中对于短序列的精度

带双向时间增强的 Transformer 序列推荐

Sequential Recommendation with Bidirectional Chronological Augmentation  of Transformer

Online sequence prediction is the problem of predicting the next element of a
sequence given previous elements. This problem has been extensively studied in
the context of individual sequence prediction, where no prior assumptions are
made on the origin of the sequence. Individual sequence prediction algorithms
work quite well for long sequences, where the algorithm has enough time to
learn the temporal structure of the sequence. However, they might give poor
predictions for short sequences. A possible remedy is to rely on the general
model of prediction with expert advice, where the learner has access to a set
of $r$ experts, each of which makes its own predictions on the sequence. It is
well known that it is possible to predict almost as well as the best expert if
the sequence length is order of $\log(r)$. But, without firm prior knowledge on
the problem, it is not clear how to choose a small set of {\em good} experts.
In this paper we describe and analyze a new algorithm that learns a good set of
experts using a training set of previously observed sequences. We demonstrate
the merits of our approach by applying it on the task of click prediction on
the web.

本文介绍一种在线学习算法，利用先前观察到的数据训练一组有效的专家，以在短序列上进行点击预测。