Given the widespread adoption of depth-sensing acquisition devices, RGB-D
videos and related data/media have gained considerable traction in various
aspects of daily life. Consequently, conducting salient object detection (SOD)
in RGB-D videos presents a highly promising and evolving avenue. Despite the
potential of this area, SOD in RGB-D videos remains somewhat under-explored,
with RGB-D SOD and video SOD (VSOD) traditionally studied in isolation. To
explore this emerging field, this paper makes two primary contributions: the
dataset and the model. On one front, we construct the RDVS dataset, a new RGB-D
VSOD dataset with realistic depth and characterized by its diversity of scenes
and rigorous frame-by-frame annotations. We validate the dataset through
comprehensive attribute and object-oriented analyses, and provide training and
testing splits. Moreover, we introduce DCTNet+, a three-stream network tailored
for RGB-D VSOD, with an emphasis on RGB modality and treats depth and optical
flow as auxiliary modalities. In pursuit of effective feature enhancement,
refinement, and fusion for precise final prediction, we propose two modules:
the multi-modal attention module (MAM) and the refinement fusion module (RFM).
To enhance interaction and fusion within RFM, we design a universal interaction
module (UIM) and then integrate holistic multi-modal attentive paths (HMAPs)
for refining multi-modal low-level features before reaching RFMs. Comprehensive
experiments, conducted on pseudo RGB-D video datasets alongside our RDVS,
highlight the superiority of DCTNet+ over 17 VSOD models and 14 RGB-D SOD
models. Ablation experiments were performed on both pseudo and realistic RGB-D
video datasets to demonstrate the advantages of individual modules as well as
the necessity of introducing realistic depth. Our code together with RDVS
dataset will be available at this https URL

通过构建 RDVS 数据集和引入 DCTNet+，本文在 RGB-D 视频中进行了显著目标检测，通过使用多模态注意力模块实现了多模态特征的增强与融合，实验结果表明 DCTNet + 在 17 个 VSOD 模型和 14 个 RGB-D SOD 模型中表现优越。