Human perception involves discerning complex multi-object scenes into
time-static object appearance (\ie, size, shape, color) and time-varying object
motion (\ie, location, velocity, acceleration). This innate ability to
unconsciously understand the environment is the motivation behind the success
of dynamics modeling. Object-centric representations have emerged as a
promising tool for dynamics prediction, yet they primarily focus on the
objects' appearance, often overlooking other crucial attributes. In this paper,
we propose Object-Centric Kinematics (OCK), a framework for dynamics prediction
leveraging object-centric representations. Our model utilizes a novel component
named object kinematics, which comprises low-level structured states of
objects' position, velocity, and acceleration. The object kinematics are
obtained via either implicit or explicit approaches, enabling comprehensive
spatiotemporal object reasoning, and integrated through various transformer
mechanisms, facilitating effective object-centric dynamics modeling. Our model
demonstrates superior performance when handling objects and backgrounds in
complex scenes characterized by a wide range of object attributes and dynamic
movements. Moreover, our model demonstrates generalization capabilities across
diverse synthetic environments, highlighting its potential for broad
applicability in vision-related tasks.

我们提出了一种名为 Object-Centric Kinematics (OCK) 的动态预测框架，利用了对象为中心的表示方法，并通过各种转换机制进行综合，以实现有效的对象为中心的动态建模。该模型在处理复杂场景中的对象和背景时表现出卓越的性能，同时在不同的合成环境中展示了泛化能力，突显了它在与视觉相关的任务中的潜力。

无监督物体本体运动动力学预测

Unsupervised Dynamics Prediction with Object-Centric Kinematics

Current Deep Learning methods for environment segmentation and velocity
estimation rely on Convolutional Recurrent Neural Networks to exploit
spatio-temporal relationships within obtained sensor data. These approaches
derive scene dynamics implicitly by correlating novel input and memorized data
utilizing ConvNets. We show how ConvNets suffer from architectural restrictions
for this task. Based on these findings, we then provide solutions to various
issues on exploiting spatio-temporal correlations in a sequence of sensor
recordings by presenting a novel Recurrent Neural Network unit utilizing
Transformer mechanisms. Within this unit, object encodings are tracked across
consecutive frames by correlating key-query pairs derived from sensor inputs
and memory states, respectively. We then use resulting tracking patterns to
obtain scene dynamics and regress velocities. In a last step, the memory state
of the Recurrent Neural Network is projected based on extracted velocity
estimates to resolve aforementioned spatio-temporal misalignment.

本研究提出了一种基于 Transformer 机制的新型循环神经网络单元，该单元能够利用序列传感器录音中的时空相关性，并跟踪对象编码，以获取场景动态和回归速度，并根据提取的速度估计将循环神经网络的内存状态进行投影，以解决空间和时间不对齐的问题。