Salient object detection (SOD) in panoramic video is still in the initial
exploration stage. The indirect application of 2D video SOD method to the
detection of salient objects in panoramic video has many unmet challenges, such
as low detection accuracy, high model complexity, and poor generalization
performance. To overcome these hurdles, we design an Inter-Layer Attention
(ILA) module, an Inter-Layer weight (ILW) module, and a Bi-Modal Attention
(BMA) module. Based on these modules, we propose a Spatial-Temporal Dual-Mode
Mixed Flow Network (STDMMF-Net) that exploits the spatial flow of panoramic
video and the corresponding optical flow for SOD. First, the ILA module
calculates the attention between adjacent level features of consecutive frames
of panoramic video to improve the accuracy of extracting salient object
features from the spatial flow. Then, the ILW module quantifies the salient
object information contained in the features of each level to improve the
fusion efficiency of the features of each level in the mixed flow. Finally, the
BMA module improves the detection accuracy of STDMMF-Net. A large number of
subjective and objective experimental results testify that the proposed method
demonstrates better detection accuracy than the state-of-the-art (SOTA)
methods. Moreover, the comprehensive performance of the proposed method is
better in terms of memory required for model inference, testing time,
complexity, and generalization performance.

通过引入 Inter-Layer Attention 模块、Inter-Layer weight 模块和 Bi-Modal Attention 模块，本研究提出了一种 Spatial-Temporal Dual-Mode Mixed Flow 网络（STDMMF-Net），该网络利用全景视频的空间流和对应的光流来实现显著目标检测。实验证明，与最先进的方法相比，该方法在检测准确性、模型推理所需内存、测试时间、复杂性和泛化能力等方面表现更好。