We present a novel embedding approach for video instance segmentation. Our
method learns a spatio-temporal embedding integrating cues from appearance,
motion, and geometry; a 3D causal convolutional network models motion, and a
monocular self-supervised depth loss models geometry. In this embedding space,
video-pixels of the same instance are clustered together while being separated
from other instances, to naturally track instances over time without any
complex post-processing. Our network runs in real-time as our architecture is
entirely causal - we do not incorporate information from future frames,
contrary to previous methods. We show that our model can accurately track and
segment instances, even with occlusions and missed detections, advancing the
state-of-the-art on the KITTI Multi-Object and Tracking Dataset.

本研究提出了一种新颖的视频实例分割嵌入式方法，该方法通过融合外观、运动和几何等多种线索来学习时空嵌入，其中，运动由 3D 因果卷积网络模拟，几何由单目自监督深度损失模拟。在这种嵌入空间中，同一实例的视频像素被聚类在一起，同时与其他实例分离，从而自然地跟踪实例而不需要任何复杂的后处理，而且实时性能良好。研究结果表明，该模型在 KITTI 多物体和跟踪数据集上可以准确地跟踪和分割实例，即使存在遮挡和遗漏检测等问题，也能推进同类算法的发展。