Event perception tasks such as recognizing and localizing actions in
streaming videos are essential for scaling to real-world application contexts.
We tackle the problem of learning actor-centered representations through the
notion of continual hierarchical predictive learning to localize actions in
streaming videos without the need for training labels and outlines for the
objects in the video. We propose a framework driven by the notion of
hierarchical predictive learning to construct actor-centered features by
attention-based contextualization. The key idea is that predictable features or
objects do not attract attention and hence do not contribute to the action of
interest. Experiments on three benchmark datasets show that the approach can
learn robust representations for localizing actions using only one epoch of
training, i.e., a single pass through the streaming video. We show that the
proposed approach outperforms unsupervised and weakly supervised baselines
while offering competitive performance to fully supervised approaches.
Additionally, we extend the model to multi-actor settings to recognize group
activities while localizing the multiple, plausible actors. We also show that
it generalizes to out-of-domain data with limited performance degradation.

在不需要训练标签和轮廓的情况下，通过连续的分层预测学习构建基于注意力的上下文特征，以学习在流媒体视频中本地化动作的鲁棒表示方法。