This paper presents a novel framework in which video/image segmentation and localization are cast into a single optimization problem that integrates information from low level appearance cues with that of high level localization cues in a very weakly supervised manner. The proposed framework leverages two representations at different levels, exploits the spatial relationship between bounding boxes and superpixels as linear constraints and simultaneously discriminates between foreground and background at bounding box and superpixel level. Different from previous approaches that mainly rely on discriminative clustering, we incorporate a foreground model that minimizes the histogram difference of an object across all image frames. Exploiting the geometric relation between the superpixels and bounding boxes enables the transfer of segmentation cues to improve localization output and vice-versa. Inclusion of the foreground model generalizes our discriminative framework to video data where the background tends to be similar and thus, not discriminative. We demonstrate the effectiveness of our unified framework on the YouTube Object video dataset, Internet Object Discovery dataset and Pascal VOC 2007.

本文提出了一种新的框架，将视频/图像分割和区域定位视为一个单一的优化问题，并以非常弱的监督方式集成低级外观线索和高级定位线索的信息。该框架利用不同级别的两种表示，利用边界框和超像素之间的空间关系作为线性约束，同时在边界框和超像素级别上区分前景和背景。与以往主要依赖于判别聚类的方法不同，我们结合了一个前景模型，使得一个物体在所有图像帧中的直方图差异最小。利用超像素和边界框之间的几何关系使得分割线索可以传递以改善定位输出，反之亦然。前景模型的引入将我们的判别框架推广到视频数据，其中背景往往是相似的，因此不具有判别力。我们证明了我们的统一框架在YouTube对象视频数据集，Internet对象发现数据集和Pascal VOC 2007上的有效性。

视频和图像联合分割与定位的前景聚类