Recently, implicit neural representations (INR) have made significant strides
in various vision-related domains, providing a novel solution for Multispectral
and Hyperspectral Image Fusion (MHIF) tasks. However, INR is prone to losing
high-frequency information and is confined to the lack of global perceptual
capabilities. To address these issues, this paper introduces a Fourier-enhanced
Implicit Neural Fusion Network (FeINFN) specifically designed for MHIF task,
targeting the following phenomena: The Fourier amplitudes of the HR-HSI latent
code and LR-HSI are remarkably similar; however, their phases exhibit different
patterns. In FeINFN, we innovatively propose a spatial and frequency implicit
fusion function (Spa-Fre IFF), helping INR capture high-frequency information
and expanding the receptive field. Besides, a new decoder employing a complex
Gabor wavelet activation function, called Spatial-Frequency Interactive Decoder
(SFID), is invented to enhance the interaction of INR features. Especially, we
further theoretically prove that the Gabor wavelet activation possesses a
time-frequency tightness property that favors learning the optimal bandwidths
in the decoder. Experiments on two benchmark MHIF datasets verify the
state-of-the-art (SOTA) performance of the proposed method, both visually and
quantitatively. Also, ablation studies demonstrate the mentioned contributions.
The code will be available on Anonymous GitHub
(this https URL) after possible acceptance.

该论文介绍了一种特定设计用于多光谱和高光谱图像融合任务的傅里叶增强的隐式神经融合网络，通过创新地提出了一种空间和频率隐式融合函数（Spa-Fre IFF）来捕捉高频信息并扩展感受野，并发明了一种新的解码器，使用复数 Gabor 小波激活函数，称为空间频率交互解码器（SFID），以增强隐式神经表示特征的交互作用。在两个基准多光谱和高光谱图像融合数据集上的实验证明了该方法在视觉上和定量上的最先进表现，同时消融研究证明了所提出的贡献。

基于傅里叶增强的隐式神经融合网络用于多光谱与高光谱图像融合

Fourier-enhanced Implicit Neural Fusion Network for Multispectral and  Hyperspectral Image Fusion

Multispectral and Hyperspectral Image Fusion (MHIF) is a practical task that
aims to fuse a high-resolution multispectral image (HR-MSI) and a
low-resolution hyperspectral image (LR-HSI) of the same scene to obtain a
high-resolution hyperspectral image (HR-HSI). Benefiting from powerful
inductive bias capability, CNN-based methods have achieved great success in the
MHIF task. However, they lack certain interpretability and require convolution
structures be stacked to enhance performance. Recently, Implicit Neural
Representation (INR) has achieved good performance and interpretability in 2D
tasks due to its ability to locally interpolate samples and utilize multimodal
content such as pixels and coordinates. Although INR-based approaches show
promise, they require extra construction of high-frequency information
(\emph{e.g.,} positional encoding). In this paper, inspired by previous work of
MHIF task, we realize that HR-MSI could serve as a high-frequency detail
auxiliary input, leading us to propose a novel INR-based hyperspectral fusion
function named Implicit Neural Feature Fusion Function (INF). As an elaborate
structure, it solves the MHIF task and addresses deficiencies in the INR-based
approaches. Specifically, our INF designs a Dual High-Frequency Fusion (DHFF)
structure that obtains high-frequency information twice from HR-MSI and LR-HSI,
then subtly fuses them with coordinate information. Moreover, the proposed INF
incorporates a parameter-free method named INR with cosine similarity (INR-CS)
that uses cosine similarity to generate local weights through feature vectors.
Based on INF, we construct an Implicit Neural Fusion Network (INFN) that
achieves state-of-the-art performance for MHIF tasks of two public datasets,
\emph{i.e.,} CAVE and Harvard. The code will soon be made available on GitHub.

本文提出了一种名为隐式神经特征融合函数（INF）的 INR（Implicit Neural Representation）型高光谱融合模型，以高频细节辅助输入来解决 MHIF（Multispectral and Hyperspectral Image Fusion）问题并且有效改进了其他 INR 的方法及提高了模型的鲁棒性。