In this paper, we propose a novel learning-based polygonal point set tracking method. Compared to existing video object segmentation~(VOS) methods that propagate pixel-wise object mask information, we propagate a polygonal point set over frames. Specifically, the set is defined as a subset of points in the target contour, and our goal is to track corresponding points on the target contour. Those outputs enable us to apply various visual effects such as motion tracking, part deformation, and texture mapping. To this end, we propose a new method to track the corresponding points between frames by the global-local alignment with delicately designed losses and regularization terms. We also introduce a novel learning strategy using synthetic and VOS datasets that makes it possible to tackle the problem without developing the point correspondence dataset. Since the existing datasets are not suitable to validate our method, we build a new polygonal point set tracking dataset and demonstrate the superior performance of our method over the baselines and existing contour-based VOS methods. In addition, we present visual-effects applications of our method on part distortion and text mapping.

本文提出了一种基于学习的多边形点集跟踪方法，通过全局-局部对齐和精细设计的损失和正则化项，以多边形点集方式传播目标轮廓，从而实现运动跟踪、部分变形和纹理映射等多种视觉效果。同时通过VOS数据集的学习策略，避免了点对应数据集的开发，并在新的跟踪数据集上证明了该方法的优异性能。