We study learning object segmentation from unlabeled videos. Humans can
easily segment moving objects without knowing what they are. The Gestalt law of
common fate, i.e., what move at the same speed belong together, has inspired
unsupervised object discovery based on motion segmentation. However, common
fate is not a reliable indicator of objectness: Parts of an articulated /
deformable object may not move at the same speed, whereas shadows / reflections
of an object always move with it but are not part of it.
Our insight is to bootstrap objectness by first learning image features from
relaxed common fate and then refining them based on visual appearance grouping
within the image itself and across images statistically. Specifically, we learn
an image segmenter first in the loop of approximating optical flow with
constant segment flow plus small within-segment residual flow, and then by
refining it for more coherent appearance and statistical figure-ground
relevance.
On unsupervised video object segmentation, using only ResNet and
convolutional heads, our model surpasses the state-of-the-art by absolute gains
of 7/9/5% on DAVIS16 / STv2 / FBMS59 respectively, demonstrating the
effectiveness of our ideas. Our code is publicly available.

本研究探讨了从无标签视频中学习物体分割的方法：首先从视觉共同命运与物体出现统计上进行特征学习，再基于视频内部和跨视频的外观分组进行细化，应用 ResNet 和卷积头只需无监督学习即可实现视频目标分割，相对于 DAVIS16 / STv2 / FBMS59 等基准上，绝对增益分别为 7/9/5％，展示了该方法的有效性，代码也已公开发表。