Existing multiple modality fusion methods, such as concatenation, summation,
and encoder-decoder-based fusion, have recently been employed to combine
modality characteristics of Hyperspectral Image (HSI) and Light Detection And
Ranging (LiDAR). However, these methods consider the relationship of HSI-LiDAR
signals from limited perspectives. More specifically, they overlook the
contextual information across modalities of HSI and LiDAR and the
intra-modality characteristics of LiDAR. In this paper, we provide a new
insight into feature fusion to explore the relationships across HSI and LiDAR
modalities comprehensively. An Interconnected Fusion (IF) framework is
proposed. Firstly, the center patch of the HSI input is extracted and
replicated to the size of the HSI input. Then, nine different perspectives in
the fusion matrix are generated by calculating self-attention and
cross-attention among the replicated center patch, HSI input, and corresponding
LiDAR input. In this way, the intra- and inter-modality characteristics can be
fully exploited, and contextual information is considered in both
intra-modality and inter-modality manner. These nine interrelated elements in
the fusion matrix can complement each other and eliminate biases, which can
generate a multi-modality representation for classification accurately.
Extensive experiments have been conducted on three widely used datasets:
Trento, MUUFL, and Houston. The IF framework achieves state-of-the-art results
on these datasets compared to existing approaches.

介绍了一种新的 “互连融合” 框架用于超光谱图像和激光雷达数据的特征融合，包括从多个视角中利用 HIS-LiDAR 信号的上下文信息和内部特征，可以提高分类准确性。