Human perception involves discerning complex multi-object scenes into
time-static object appearance (\ie, size, shape, color) and time-varying object
motion (\ie, location, velocity, acceleration). This innate ability to
unconsciously understand the environment is the motivation behind the success
of dynamics modeling. Object-centric representations have emerged as a
promising tool for dynamics prediction, yet they primarily focus on the
objects' appearance, often overlooking other crucial attributes. In this paper,
we propose Object-Centric Kinematics (OCK), a framework for dynamics prediction
leveraging object-centric representations. Our model utilizes a novel component
named object kinematics, which comprises low-level structured states of
objects' position, velocity, and acceleration. The object kinematics are
obtained via either implicit or explicit approaches, enabling comprehensive
spatiotemporal object reasoning, and integrated through various transformer
mechanisms, facilitating effective object-centric dynamics modeling. Our model
demonstrates superior performance when handling objects and backgrounds in
complex scenes characterized by a wide range of object attributes and dynamic
movements. Moreover, our model demonstrates generalization capabilities across
diverse synthetic environments, highlighting its potential for broad
applicability in vision-related tasks.

我们提出了一种名为 Object-Centric Kinematics (OCK) 的动态预测框架，利用了对象为中心的表示方法，并通过各种转换机制进行综合，以实现有效的对象为中心的动态建模。该模型在处理复杂场景中的对象和背景时表现出卓越的性能，同时在不同的合成环境中展示了泛化能力，突显了它在与视觉相关的任务中的潜力。

无监督物体本体运动动力学预测

Unsupervised Dynamics Prediction with Object-Centric Kinematics

How would a static scene react to a local poke? What are the effects on other
parts of an object if you could locally push it? There will be distinctive
movement, despite evident variations caused by the stochastic nature of our
world. These outcomes are governed by the characteristic kinematics of objects
that dictate their overall motion caused by a local interaction. Conversely,
the movement of an object provides crucial information about its underlying
distinctive kinematics and the interdependencies between its parts. This
two-way relation motivates learning a bijective mapping between object
kinematics and plausible future image sequences. Therefore, we propose iPOKE --
invertible Prediction of Object Kinematics -- that, conditioned on an initial
frame and a local poke, allows to sample object kinematics and establishes a
one-to-one correspondence to the corresponding plausible videos, thereby
providing a controlled stochastic video synthesis. In contrast to previous
works, we do not generate arbitrary realistic videos, but provide efficient
control of movements, while still capturing the stochastic nature of our
environment and the diversity of plausible outcomes it entails. Moreover, our
approach can transfer kinematics onto novel object instances and is not
confined to particular object classes. Our project page is available at
this https URL

本文介绍了 iPOKE 的方法，它基于一种双向匹配的思想，计算出对象的运动与未来的图像序列之间的关系，并能够建立对应的图像，实现了对对象的控制运动。