Video Object Segmentation (VOS) is fundamental to video understanding. Transformer-based methods show significant performance improvement on semi-supervised VOS. However, existing work faces challenges segmenting visually similar objects in close proximity of each other. In this paper, we propose a novel Bilateral Attention Transformer in Motion-Appearance Neighboring space (BATMAN) for semi-supervised VOS. It captures object motion in the video via a novel optical flow calibration module that fuses the segmentation mask with optical flow estimation to improve within-object optical flow smoothness and reduce noise at object boundaries. This calibrated optical flow is then employed in our novel bilateral attention, which computes the correspondence between the query and reference frames in the neighboring bilateral space considering both motion and appearance. Extensive experiments validate the effectiveness of BATMAN architecture by outperforming all existing state-of-the-art on all four popular VOS benchmarks: Youtube-VOS 2019 (85.0%), Youtube-VOS 2018 (85.3%), DAVIS 2017Val/Testdev (86.2%/82.2%), and DAVIS 2016 (92.5%).

在这篇文章中，作者们提出了一个基于Bilateral Attention Transformer的新型半监督Video Object Segmentation算法，该算法通过光流校准模块来捕捉视频中的物体运动，从而降低物体边界噪音，并利用邻域双边空间中的对应关系计算查询和参考帧之间的对应关系，从而提高物体分割效果。该算法在所有四个流行的VOS基准测试（Youtube-VOS 2019，Youtube-VOS 2018，DAVIS 2017Val/Testdev和DAVIS 2016）上都比现有的半监督VOS算法表现更优。

BATMAN: 双边注意力变换器用于运动-外观邻域空间的视频对象分割