Any human activity can be represented as a temporal sequence of actions
performed to achieve a certain goal. Unlike machine-made time series, these
action sequences are highly disparate as the time taken to finish a similar
action might vary between different persons. Therefore, understanding the
dynamics of these sequences is essential for many downstream tasks such as
activity length prediction, goal prediction, etc. Existing neural approaches
that model an activity sequence are either limited to visual data or are task
specific, i.e., limited to next action or goal prediction. In this paper, we
present ProActive, a neural marked temporal point process (MTPP) framework for
modeling the continuous-time distribution of actions in an activity sequence
while simultaneously addressing three high-impact problems -- next action
prediction, sequence-goal prediction, and end-to-end sequence generation.
Specifically, we utilize a self-attention module with temporal normalizing
flows to model the influence and the inter-arrival times between actions in a
sequence. Moreover, for time-sensitive prediction, we perform an early
detection of sequence goal via a constrained margin-based optimization
procedure. This in-turn allows ProActive to predict the sequence goal using a
limited number of actions. Extensive experiments on sequences derived from
three activity recognition datasets show the significant accuracy boost of
ProActive over the state-of-the-art in terms of action and goal prediction, and
the first-ever application of end-to-end action sequence generation.

ProActive 是一种神经标记的时间点过程 (MTPP) 框架，用于对活动序列中行动的连续时间分布进行建模，同时解决了下一步行动预测，序列目标预测和端到端序列生成等高影响问题。使用自关注模块进行建模，以及附带较少的行动数量来预测序列目标。在三个活动识别数据集上的广泛实验表明，ProActive 在行动和目标预测方面的准确性显着提高，并首次应用了端到端行动序列生成。