Unsupervised video object segmentation (VOS) is a task that aims to detect
the most salient object in a video without external guidance about the object.
To leverage the property that salient objects usually have distinctive
movements compared to the background, recent methods collaboratively use motion
cues extracted from optical flow maps with appearance cues extracted from RGB
images. However, as optical flow maps are usually very relevant to segmentation
masks, the network is easy to be learned overly dependent on the motion cues
during network training. As a result, such two-stream approaches are vulnerable
to confusing motion cues, making their prediction unstable. To relieve this
issue, we design a novel motion-as-option network by treating motion cues as
optional. During network training, RGB images are randomly provided to the
motion encoder instead of optical flow maps, to implicitly reduce motion
dependency of the network. As the learned motion encoder can deal with both RGB
images and optical flow maps, two different predictions can be generated
depending on which source information is used as motion input. In order to
fully exploit this property, we also propose an adaptive output selection
algorithm to adopt optimal prediction result at test time. Our proposed
approach affords state-of-the-art performance on all public benchmark datasets,
even maintaining real-time inference speed.

无监督视频对象分割是一项旨在在没有关于对象的外部指导的情况下检测视频中最显著的对象的任务。为了充分利用显著对象通常与背景具有不同的运动特征的属性，最近的方法共同使用从光流图提取的运动线索与从 RGB 图像提取的外观线索。然而，由于光流图通常与分割掩码非常相关，网络在网络训练期间容易过度依赖运动线索。因此，这种两流方法容易受到混淆的运动线索的干扰，使其预测不稳定。为了缓解这个问题，我们设计了一种将运动线索视为可选的新型运动作为选项网络。在网络训练期间，随机提供 RGB 图像而不是光流图给运动编码器，以隐式降低网络的运动依赖性。由于学习到的运动编码器可以处理 RGB 图像和光流图，因此可以根据所使用的信息源的运动输入生成两种不同的预测。为了充分利用这个特性，我们还提出了一种自适应输出选择算法来在测试时采用最佳的预测结果。我们提出的方法在所有公共基准数据集上都具有最先进的性能，甚至保持了实时推理速度。

无监督视频目标分割中以输出选择为选项处理动作

Treating Motion as Option with Output Selection for Unsupervised Video  Object Segmentation

Computational saliency models for still images have gained significant
popularity in recent years. Saliency prediction from videos, on the other hand,
has received relatively little interest from the community. Motivated by this,
in this work, we study the use of deep learning for dynamic saliency prediction
and propose the so-called spatio-temporal saliency networks. The key to our
models is the architecture of two-stream networks where we investigate
different fusion mechanisms to integrate spatial and temporal information. We
evaluate our models on the DIEM and UCF-Sports datasets and present highly
competitive results against the existing state-of-the-art models. We also carry
out some experiments on a number of still images from the MIT300 dataset by
exploiting the optical flow maps predicted from these images. Our results show
that considering inherent motion information in this way can be helpful for
static saliency estimation.

本研究提出了基于深度学习的时空注意力预测模型，其中采用两条流网络架构，探究了不同的信息融合机制。实验结果表明，采用运动信息可以有助于静态注意力估计，并取得了与最先进模型相竞争的结果。