We consider the problem of segmenting objects in videos based on their motion and no other forms of supervision. Prior work has often approached this problem by using the principle of common fate, namely the fact that the motion of points that belong to the same object is strongly correlated. However, most authors have only considered instantaneous motion from optical flow. In this work, we present a way to train a segmentation network using long-term point trajectories as a supervisory signal to complement optical flow. The key difficulty is that long-term motion, unlike instantaneous motion, is difficult to model -- any parametric approximation is unlikely to capture complex motion patterns over long periods of time. We instead draw inspiration from subspace clustering approaches, proposing a loss function that seeks to group the trajectories into low-rank matrices where the motion of object points can be approximately explained as a linear combination of other point tracks. Our method outperforms the prior art on motion-based segmentation, which shows the utility of long-term motion and the effectiveness of our formulation.

本研究针对基于运动的对象分割问题，通过使用长时间点轨迹作为监督信号来补充光流，解决了传统方法过于依赖瞬时运动的问题。研究提出了一种新的损失函数，旨在将轨迹分组为低秩矩阵，从而有效建模复杂的长时间运动模式，并在运动分割任务上超越了现有方法，显示了长期运动的应用潜力。

从点轨迹学习分割