Existing RGB-D salient object detection (SOD) approaches concentrate on the
cross-modal fusion between the RGB stream and the depth stream. They do not
deeply explore the effect of the depth map itself. In this work, we design a
single stream network to directly use the depth map to guide early fusion and
middle fusion between RGB and depth, which saves the feature encoder of the
depth stream and achieves a lightweight and real-time model. We tactfully
utilize depth information from two perspectives: (1) Overcoming the
incompatibility problem caused by the great difference between modalities, we
build a single stream encoder to achieve the early fusion, which can take full
advantage of ImageNet pre-trained backbone model to extract rich and
discriminative features. (2) We design a novel depth-enhanced dual attention
module (DEDA) to efficiently provide the fore-/back-ground branches with the
spatially filtered features, which enables the decoder to optimally perform the
middle fusion. Besides, we put forward a pyramidally attended feature
extraction module (PAFE) to accurately localize the objects of different
scales. Extensive experiments demonstrate that the proposed model performs
favorably against most state-of-the-art methods under different evaluation
metrics. Furthermore, this model is 55.5\% lighter than the current lightest
model and runs at a real-time speed of 32 FPS when processing a $384 \times
384$ image.

本研究提出一种单流网络，利用深度信息实现早期融合和中期融合之间的引导，解决不同模态之间的不兼容问题，并设计了一种增强深度双重注意力模块（DEDA）和锥形局部注意提取模块（PAFE），以提高实时目标检测的准确性和效率。

一种单流网络的鲁棒性和实时性 RGB-D 显著性目标检测

A Single Stream Network for Robust and Real-time RGB-D Salient Object  Detection

Current cross-modal retrieval systems are evaluated using R@K measure which
does not leverage semantic relationships rather strictly follows the manually
marked image text query pairs. Therefore, current systems do not generalize
well for the unseen data in the wild. To handle this, we propose a new measure,
SemanticMap, to evaluate the performance of cross-modal systems. Our proposed
measure evaluates the semantic similarity between the image and text
representations in the latent embedding space. We also propose a novel
cross-modal retrieval system using a single stream network for bidirectional
retrieval. The proposed system is based on a deep neural network trained using
extended center loss, minimizing the distance of image and text descriptions in
the latent space from the class centers. In our system, the text descriptions
are also encoded as images which enabled us to use a single stream network for
both text and images. To the best of our knowledge, our work is the first of
its kind in terms of employing a single stream network for cross-modal
retrieval systems. The proposed system is evaluated on two publicly available
datasets including MSCOCO and Flickr30K and has shown comparable results to the
current state-of-the-art methods.

该论文提出一种新的语义相似度评估指标 SemanticMap 来评估跨模态检索系统的性能，并且提出了基于单流网络的新型跨模态检索系统，该系统使用深度神经网络训练，并在 MSCOCO 和 Flickr30K 数据集上进行了评估。