Thanks to the development of basic models, infrared small target detection
(ISTD) algorithms have made significant progress. Specifically, the structures
combining convolutional networks with transformers can well extract both local
and global features. At the same time, they also inherit defects from the basic
model, e.g., the quadratic computational complexity of transformers, which
impacts efficiency. Inspired by a recent basic model with linear complexity for
long-distance modeling, called Mamba, we explore the potential of this state
space model in ISTD in this paper. However, direct application is unsuitable
since local features, which are critical to detecting small targets, cannot be
fully exploited. Instead, we tailor a Mamba-in-Mamba (MiM-ISTD) structure for
efficient ISTD. For example, we treat the local patches as "visual sentences"
and further decompose them into sub-patches as "visual words" to further
explore the locality. The interactions among each word in a given visual
sentence will be calculated with negligible computational costs. By aggregating
the word and sentence features, the representation ability of MiM-ISTD can be
significantly bolstered. Experiments on NUAA-SIRST and IRSTD-1k prove the
superior accuracy and efficiency of our method. Specifically, MiM-ISTD is $10
\times$ faster than the SOTA and reduces GPU memory usage by 73.4$\%$ per $2048
\times 2048$ image during inference, overcoming the computation$\&$memory
constraints on performing Mamba-based understanding on high-resolution infrared
images.Source code is available at this https URL

通过将卷积网络与 Transformer 相结合的结构，本文探索了一种用于高效红外小目标检测的 Mamba-in-Mamba 结构，该结构通过对局部块进行子块分解，利用每个子块之间的相互作用来大大提升 MiM-ISTD 的表征能力和计算效率。实验证明，MiM-ISTD 方法的准确性和效率均优于现有方法，其推理速度比 SOTA 快 10 倍，并且在推理过程中减少了 73.4% 的 GPU 内存使用量，克服了在高分辨率红外图像上执行基于 Mamba 的理解时的计算和内存限制。