This paper presents a novel method to predict future human activities from
partially observed RGB-D videos. Human activity prediction is generally
difficult due to its non-Markovian property and the rich context between human
and environments.
We use a stochastic grammar model to capture the compositional structure of
events, integrating human actions, objects, and their affordances. We represent
the event by a spatial-temporal And-Or graph (ST-AOG). The ST-AOG is composed
of a temporal stochastic grammar defined on sub-activities, and spatial graphs
representing sub-activities that consist of human actions, objects, and their
affordances. Future sub-activities are predicted using the temporal grammar and
Earley parsing algorithm. The corresponding action, object, and affordance
labels are then inferred accordingly. Extensive experiments are conducted to
show the effectiveness of our model on both semantic event parsing and future
activity prediction.

本文提出了一种新的方法来从部分观察到的 RGB-D 视频中预测未来的人类活动，使用随机语法模型来捕捉事件的组成结构，作为一个空间时间 And-Or 图 (ST-AOG)，未来的子活动通过 Earley 解析算法预测，我们通过广泛的实验验证了我们模型在语义事件标记和未来活动预测方面的有效性。