Identifying the land cover category for each pixel in a hyperspectral image
(HSI) relies on spectral and spatial information. An HSI cuboid with a specific
patch size is utilized to extract spatial-spectral feature representation for
the central pixel. In this article, we investigate that scene-specific but not
essential correlations may be recorded in an HSI cuboid. This additional
information improves the model performance on existing HSI datasets and makes
it hard to properly evaluate the ability of a model. We refer to this problem
as the spatial overfitting issue and utilize strict experimental settings to
avoid it. We further propose a multiview transformer for HSI classification,
which consists of multiview principal component analysis (MPCA), spectral
encoder-decoder (SED), and spatial-pooling tokenization transformer (SPTT).
MPCA performs dimension reduction on an HSI via constructing spectral multiview
observations and applying PCA on each view data to extract low-dimensional view
representation. The combination of view representations, named multiview
representation, is the dimension reduction output of the MPCA. To aggregate the
multiview information, a fully-convolutional SED with a U-shape in spectral
dimension is introduced to extract a multiview feature map. SPTT transforms the
multiview features into tokens using the spatial-pooling tokenization strategy
and learns robust and discriminative spatial-spectral features for land cover
identification. Classification is conducted with a linear classifier.
Experiments on three HSI datasets with rigid settings demonstrate the
superiority of the proposed multiview transformer over the state-of-the-art
methods.

通过使用具有特定块大小的 HSI 立方体提取中心像素的空间 - 光谱特征表示，本研究论文探讨了 HSI 立方体中可能记录到的场景特定但非必要的相关性，这些附加信息改善了现有 HSI 数据集上模型的性能，并使其难以正确评估模型的能力。作者引入空间过拟合问题，并利用严格的实验设置来避免该问题。除此之外，论文还提出了一种用于 HSI 分类的多视图变换器，包括多视图主成分分析（MPCA）、光谱编码器 - 解码器（SED）和空间池化标记化变形器（SPTT）。通过构建光谱多视图观察并在每个视图数据上应用 PCA 来降维 HSI，MPCA 能够提取低维视图表示，这些视图表示的组合被称为多视图表示，并作为 MPCA 的输出进行传递。为了聚合多视图信息，论文引入了具有在光谱维度上呈 U 形状的全卷积 SED，用于提取多视图特征图。SPTT 利用空间池化标记化策略将多视图特征转换为标记，并为土地覆盖分类学习了健壮且具有区分性的空间 - 光谱特征。分类采用线性分类器进行，实验结果表明，所提出的多视图变换器优于现有方法，经过严格设置的三个 HSI 数据集上都取得了优越的性能。

多视角变换器：超光谱图像分类中空间信息的重新思考

Multiview Transformer: Rethinking Spatial Information in Hyperspectral  Image Classification

Recently, graph convolutional networks (GCNs) have been developed to explore
spatial relationship between pixels, achieving better classification
performance of hyperspectral images (HSIs). However, these methods fail to
sufficiently leverage the relationship between spectral bands in HSI data. As
such, we propose an adaptive cross-attention-driven spatial-spectral graph
convolutional network (ACSS-GCN), which is composed of a spatial GCN (Sa-GCN)
subnetwork, a spectral GCN (Se-GCN) subnetwork, and a graph cross-attention
fusion module (GCAFM). Specifically, Sa-GCN and Se-GCN are proposed to extract
the spatial and spectral features by modeling correlations between spatial
pixels and between spectral bands, respectively. Then, by integrating attention
mechanism into information aggregation of graph, the GCAFM, including three
parts, i.e., spatial graph attention block, spectral graph attention block, and
fusion block, is designed to fuse the spatial and spectral features and
suppress noise interference in Sa-GCN and Se-GCN. Moreover, the idea of the
adaptive graph is introduced to explore an optimal graph through back
propagation during the training process. Experiments on two HSI data sets show
that the proposed method achieves better performance than other classification
methods.

提出了一种自适应的空间 - 光谱图卷积网络，其中包括一个空间图卷积子网络、一个光谱图卷积子网络和一个图交叉注意力融合模块，通过加入注意力机制，融合了空间和光谱特征，并探索了优化图形的思想，在两个高光谱图像数据集上实验证明了该方法比其他分类方法具有更好的性能。