Accurate video understanding involves reasoning about the relationships
between actors, objects and their environment, often over long temporal
intervals. In this paper, we propose a message passing graph neural network
that explicitly models these spatio-temporal relations and can use explicit
representations of objects, when supervision is available, and implicit
representations otherwise. Our formulation generalises previous structured
models for video understanding, and allows us to study how different design
choices in graph structure and representation affect the model's performance.
We demonstrate our method on two different tasks requiring relational reasoning
in videos -- spatio-temporal action detection on AVA and UCF101-24, and video
scene graph classification on the recent Action Genome dataset -- and achieve
state-of-the-art results on all three datasets. Furthermore, we show
quantitatively and qualitatively how our method is able to more effectively
model relationships between relevant entities in the scene.

本文提出了一种基于信息传递的图神经网络，能够精确理解视频中行为者、对象和环境之间的关系，在显式表示对象时能够使用显式表示，否则使用隐含表示；在 AVA 和 UCF101-24 的时空行为检测以及最近的 Action Genome 数据集上的视频场景图分类等任务中，实现了最先进的结果，我们定量和定性的显示了我们的方法如何更有效地模拟场景中相关实体之间的关系。

用于视频理解的统一图结构模型

Unified Graph Structured Models for Video Understanding

This study follows many classical approaches to multi-object tracking (MOT)
that model the problem using dynamic graphical data structures, and adapts this
formulation to make it amenable to modern neural networks. Our main
contributions in this work are the creation of a framework based on dynamic
undirected graphs that represent the data association problem over multiple
timesteps, and a message passing graph neural network (MPNN) that operates on
these graphs to produce the desired likelihood for every association therein.
We also provide solutions and propositions for the computational problems that
need to be addressed to create a memory-efficient, real-time, online algorithm
that can reason over multiple timesteps, correct previous mistakes, update
beliefs, and handle missed/false detections. To demonstrate the efficacy of our
approach, we only use the 2D box location and object category ID to construct
the descriptor for each object instance. Despite this, our model performs on
par with state-of-the-art approaches that make use of additional sensors, as
well as multiple hand-crafted and/or learned features. This illustrates that
given the right problem formulation and model design, raw bounding boxes (and
their kinematics) from any off-the-shelf detector are sufficient to achieve
competitive tracking results on challenging MOT benchmarks.

该研究致力于使用动态无向图和图形神经网络来解决多目标跟踪问题，同时提供了有效的解决方案和建议，证明了使用任何现成的检测器提供的原始边界框就足以在具有挑战性的多目标跟踪数据集上实现有竞争力的跟踪结果。