Many algorithms have been developed to solve the inverse problem of coded
aperture snapshot spectral imaging (CASSI), i.e., recovering the 3D
hyperspectral images (HSIs) from a 2D compressive measurement. In recent years,
learning-based methods have demonstrated promising performance and dominated
the mainstream research direction. However, existing CNN-based methods show
limitations in capturing long-range dependencies and non-local self-similarity.
Previous Transformer-based methods densely sample tokens, some of which are
uninformative, and calculate the multi-head self-attention (MSA) between some
tokens that are unrelated in content. This does not fit the spatially sparse
nature of HSI signals and limits the model scalability. In this paper, we
propose a novel Transformer-based method, coarse-to-fine sparse Transformer
(CST), firstly embedding HSI sparsity into deep learning for HSI
reconstruction. In particular, CST uses our proposed spectra-aware screening
mechanism (SASM) for coarse patch selecting. Then the selected patches are fed
into our customized spectra-aggregation hashing multi-head self-attention
(SAH-MSA) for fine pixel clustering and self-similarity capturing.
Comprehensive experiments show that our CST significantly outperforms
state-of-the-art methods while requiring cheaper computational costs. The code
and models will be released at this https URL

该论文提出了一种基于 Transformer 的 CST 方法，首次将 HSI 稀疏性嵌入到深度学习中进行重建，使用 SA-MSA 进行粗细层次的像素聚类和自相似性捕捉，相比于现有方法具有更高的重建性能和更低的计算成本。