Infrared and visible image fusion has gradually proved to be a vital fork in
the field of multi-modality imaging technologies. In recent developments,
researchers not only focus on the quality of fused images but also evaluate
their performance in downstream tasks. Nevertheless, the majority of methods
seldom put their eyes on the mutual learning from different modalities,
resulting in fused images lacking significant details and textures. To overcome
this issue, we propose an interactive graph neural network (GNN)-based
architecture between cross modality for fusion, called IGNet. Specifically, we
first apply a multi-scale extractor to achieve shallow features, which are
employed as the necessary input to build graph structures. Then, the graph
interaction module can construct the extracted intermediate features of the
infrared/visible branch into graph structures. Meanwhile, the graph structures
of two branches interact for cross-modality and semantic learning, so that
fused images can maintain the important feature expressions and enhance the
performance of downstream tasks. Besides, the proposed leader nodes can improve
information propagation in the same modality. Finally, we merge all graph
features to get the fusion result. Extensive experiments on different datasets
(TNO, MFNet and M3FD) demonstrate that our IGNet can generate visually
appealing fused images while scoring averagely 2.59% mAP@.5 and 7.77% mIoU
higher in detection and segmentation than the compared state-of-the-art
methods. The source code of the proposed IGNet can be available at
this https URL

提出了一种基于交叉模式的互动图神经网络（GNN）的架构，称为 IGNet，用于红外和可见光图像融合，通过构建图结构实现了不同模态之间的互动学习，从而提高融合图像的表达和下游任务的性能。

学习跨模态交互的图神经网络用于图像融合

Learning a Graph Neural Network with Cross Modality Interaction for  Image Fusion

Multi-intent detection and slot filling joint models are gaining increasing
traction since they are closer to complicated real-world scenarios. However,
existing approaches (1) focus on identifying implicit correlations between
utterances and one-hot encoded labels in both tasks while ignoring explicit
label characteristics; (2) directly incorporate multi-intent information for
each token, which could lead to incorrect slot prediction due to the
introduction of irrelevant intent. In this paper, we propose a framework termed
DGIF, which first leverages the semantic information of labels to give the
model additional signals and enriched priors. Then, a multi-grain interactive
graph is constructed to model correlations between intents and slots.
Specifically, we propose a novel approach to construct the interactive graph
based on the injection of label semantics, which can automatically update the
graph to better alleviate error propagation. Experimental results show that our
framework significantly outperforms existing approaches, obtaining a relative
improvement of 13.7% over the previous best model on the MixATIS dataset in
overall accuracy.

本文提出了一种名为 DGIF 的框架，采用语义标签信息构建交互式图模型来模拟意图和槽之间的关系，相较于现有方法在 MixATIS 数据集上的综合准确度相对提升了 13.7%。