The success of deep neural networks for pan-sharpening is commonly in a form
of black box, lacking transparency and interpretability. To alleviate this
issue, we propose a novel model-driven deep unfolding framework with image
reasoning prior tailored for the pan-sharpening task. Different from existing
unfolding solutions that deliver the proximal operator networks as the
uncertain and vague priors, our framework is motivated by the content reasoning
ability of masked autoencoders (MAE) with insightful designs. Specifically, the
pre-trained MAE with spatial masking strategy, acting as intrinsic reasoning
prior, is embedded into unfolding architecture. Meanwhile, the pre-trained MAE
with spatial-spectral masking strategy is treated as the regularization term
within loss function to constrain the spatial-spectral consistency. Such
designs penetrate the image reasoning prior into deep unfolding networks while
improving its interpretability and representation capability. The uniqueness of
our framework is that the holistic learning process is explicitly integrated
with the inherent physical mechanism underlying the pan-sharpening task.
Extensive experiments on multiple satellite datasets demonstrate the
superiority of our method over the existing state-of-the-art approaches. Code
will be released at https://manman1995.github.io/.

基于模型驱动的深度展开框架和图像推理先验的概念，通过嵌入空间掩蔽策略的预训练的自动编码器 (MAE) 和空间光谱掩蔽策略的正则化项，将图像推理先验融入到深度展开网络中，提高其解释性和表示能力，实现与全局学习过程和全局物理机制的显式集成，多种卫星数据集上的实验证明该方法优于现有的最先进方法。