Toward desirable saliency prediction, the types and numbers of inputs for a
salient object detection (SOD) algorithm may dynamically change in many
real-life applications. However, existing SOD algorithms are mainly designed or
trained for one particular type of inputs, failing to be generalized to other
types of inputs. Consequentially, more types of SOD algorithms need to be
prepared in advance for handling different types of inputs, raising huge
hardware and research costs. Differently, in this paper, we propose a new type
of SOD task, termed Arbitrary Modality SOD (AM SOD). The most prominent
characteristics of AM SOD are that the modality types and modality numbers will
be arbitrary or dynamically changed. The former means that the inputs to the AM
SOD algorithm may be arbitrary modalities such as RGB, depths, or even any
combination of them. While, the latter indicates that the inputs may have
arbitrary modality numbers as the input type is changed, e.g. single-modality
RGB image, dual-modality RGB-Depth (RGB-D) images or triple-modality
RGB-Depth-Thermal (RGB-D-T) images. Accordingly, a preliminary solution to the
above challenges, \i.e. a modality switch network (MSN), is proposed in this
paper. In particular, a modality switch feature extractor (MSFE) is first
designed to extract discriminative features from each modality effectively by
introducing some modality indicators, which will generate some weights for
modality switching. Subsequently, a dynamic fusion module (DFM) is proposed to
adaptively fuse features from a variable number of modalities based on a novel
Transformer structure. Finally, a new dataset, named AM-XD, is constructed to
facilitate research on AM SOD. Extensive experiments demonstrate that our AM
SOD method can effectively cope with changes in the type and number of input
modalities for robust salient object detection.

为了达到理想的显著性预测，我们提出了一种新的显著目标检测任务，被称为任意模态显著目标检测（AM SOD）。我们设计了一个调制切换网络（MSN），包括调制切换特征提取器（MSFE）和动态融合模块（DFM），用于有效地从不同模态提取鉴别特征，并自适应地融合来自不同模态的特征。此外，我们构建了一个名为 AM-XD 的新数据集，以促进对 AM SOD 的研究。广泛的实验证明，我们的方法可以有效地处理输入模态的类型和数量变化，实现鲁棒的显著目标检测。