This paper addresses a new problem of joint object boundary detection and boundary motion estimation in videos, which we named boundary flow estimation. Boundary flow is an important mid-level visual cue as boundaries characterize spatial extents of objects, and the flow indicates motions and interactions of objects. Yet, most prior work on motion estimation has focused on dense object motion or feature points that may not necessarily reside on boundaries. For boundary flow estimation, we specify a new fully convolutional Siamese network (FCSN) that jointly estimates object-level boundaries in two consecutive frames. Boundary correspondences in the two frames are predicted by the same FCSN with a new, unconventional deconvolution approach. Finally, the boundary flow estimate is improved with an edgelet based filtering. Evaluation is conducted on three tasks: boundary detection in videos, boundary flow estimation, and optical flow estimation. On boundary detection, we achieve the state-of-the-art performance on the benchmark VSB100 dataset. On boundary flow estimation, we present the first results on the Sintel training dataset. For optical flow estimation, we run the recent approach CPM-Flow but on the augmented input with our boundary flow matches, and achieve significant performance improvement on the Sintel benchmark.

本文研究采用深度学习技术解决视频中的联合物体边界检测和边界运动估计问题，提出了一种新的全卷积Siamese神经网络架构，成功实现了对物体边界的检测和运动估计。在三项任务中，本文方法分别达到了在VSB100、Sintel等数据集上的最佳性能表现。

Boundary Flow：一种用孪生网络预测边界运动的方法，无需对运动进行训练