Traditional unsupervised optical flow methods are vulnerable to occlusions and motion boundaries due to lack of object-level information. Therefore, we propose UnSAMFlow, an unsupervised flow network that also leverages object information from the latest foundation model Segment Anything Model (SAM). We first include a self-supervised semantic augmentation module tailored to SAM masks. We also analyze the poor gradient landscapes of traditional smoothness losses and propose a new smoothness definition based on homography instead. A simple yet effective mask feature module has also been added to further aggregate features on the object level. With all these adaptations, our method produces clear optical flow estimation with sharp boundaries around objects, which outperforms state-of-the-art methods on both KITTI and Sintel datasets. Our method also generalizes well across domains and runs very efficiently.

我们提出了一种利用最新的基础模型Segment Anything Model（SAM）的非监督流网络UnSAMFlow，通过加入适应SAM蒙版的自监督语义增强模块、基于单应性的新平滑度定义以及简单而有效的蒙版特征模块，能够在对象周围生成清晰的光流估计，优于KITTI和Sintel数据集上的最先进方法，并且在跨领域上具有很好的泛化能力且运行效率高。

UnSAMFlow: 由分割任意模型指导的无监督光流