Spatio-temporal scene graphs represent interactions in a video by decomposing
scenes into individual objects and their pair-wise temporal relationships.
Long-term anticipation of the fine-grained pair-wise relationships between
objects is a challenging problem. To this end, we introduce the task of Scene
Graph Anticipation (SGA). We adapt state-of-the-art scene graph generation
methods as baselines to anticipate future pair-wise relationships between
objects and propose a novel approach SceneSayer. In SceneSayer, we leverage
object-centric representations of relationships to reason about the observed
video frames and model the evolution of relationships between objects. We take
a continuous time perspective and model the latent dynamics of the evolution of
object interactions using concepts of NeuralODE and NeuralSDE, respectively. We
infer representations of future relationships by solving an Ordinary
Differential Equation and a Stochastic Differential Equation, respectively.
Extensive experimentation on the Action Genome dataset validates the efficacy
of the proposed methods.

分析了视频中的时空场景图，提出了 SceneSayer 方法，通过对观察到的视频帧进行推理，模拟对象之间关系的演化，并使用神经常微分方程和神经随机微分方程来预测对象之间的未来关系。在 Action Genome 数据集上进行了大量实验验证了该方法的效果。