Hyperspectral video (HSV) offers valuable spatial, spectral, and temporal
information simultaneously, making it highly suitable for handling challenges
such as background clutter and visual similarity in object tracking. However,
existing methods primarily focus on band regrouping and rely on RGB trackers
for feature extraction, resulting in limited exploration of spectral
information and difficulties in achieving complementary representations of
object features. In this paper, a spatial-spectral fusion network with spectral
angle awareness (SST-Net) is proposed for hyperspectral (HS) object tracking.
Firstly, to address the issue of insufficient spectral feature extraction in
existing networks, a spatial-spectral feature backbone ($S^2$FB) is designed.
With the spatial and spectral extraction branch, a joint representation of
texture and spectrum is obtained. Secondly, a spectral attention fusion module
(SAFM) is presented to capture the intra- and inter-modality correlation to
obtain the fused features from the HS and RGB modalities. It can incorporate
the visual information into the HS spectral context to form a robust
representation. Thirdly, to ensure a more accurate response of the tracker to
the object position, a spectral angle awareness module (SAAM) investigates the
region-level spectral similarity between the template and search images during
the prediction stage. Furthermore, we develop a novel spectral angle awareness
loss (SAAL) to offer guidance for the SAAM based on similar regions. Finally,
to obtain the robust tracking results, a weighted prediction method is
considered to combine the HS and RGB predicted motions of objects to leverage
the strengths of each modality. Extensive experiments on the HOTC dataset
demonstrate the effectiveness of the proposed SSF-Net, compared with
state-of-the-art trackers.

提出了一种具有光谱角度感知的空间 - 光谱融合神经网络（SST-Net），用于高光谱目标跟踪。