Recently, multi-modality scene perception tasks, e.g., image fusion and scene
understanding, have attracted widespread attention for intelligent vision
systems. However, early efforts always consider boosting a single task
unilaterally and neglecting others, seldom investigating their underlying
connections for joint promotion. To overcome these limitations, we establish
the hierarchical dual tasks-driven deep model to bridge these tasks.
Concretely, we firstly construct an image fusion module to fuse complementary
characteristics and cascade dual task-related modules, including a
discriminator for visual effects and a semantic network for feature
measurement. We provide a bi-level perspective to formulate image fusion and
follow-up downstream tasks. To incorporate distinct task-related responses for
image fusion, we consider image fusion as a primary goal and dual modules as
learnable constraints. Furthermore, we develop an efficient first-order
approximation to compute corresponding gradients and present dynamic weighted
aggregation to balance the gradients for fusion learning. Extensive experiments
demonstrate the superiority of our method, which not only produces visually
pleasant fused results but also realizes significant promotion for detection
and segmentation than the state-of-the-art approaches.

本文描述了一种层次双任务驱动的深度模型，用于多模态场景感知，包括图像融合和场景理解。该模型不仅能够产生视觉良好的融合结果，而且比现有技术实现了显著的检测和分割优势。