Recent advancements in autoregressive networks with linear complexity have
driven significant research progress, demonstrating exceptional performance in
large language models. A representative model is the Extended Long Short-Term
Memory (xLSTM), which incorporates gating mechanisms and memory structures,
performing comparably to Transformer architectures in long-sequence language
tasks. Autoregressive networks such as xLSTM can utilize image serialization to
extend their application to visual tasks such as classification and
segmentation. Although existing studies have demonstrated Vision-LSTM's
impressive results in image classification, its performance in image semantic
segmentation remains unverified. Our study represents the first attempt to
evaluate the effectiveness of Vision-LSTM in the semantic segmentation of
remotely sensed images. This evaluation is based on a specifically designed
encoder-decoder architecture named Seg-LSTM, and comparisons with
state-of-the-art segmentation networks. Our study found that Vision-LSTM's
performance in semantic segmentation was limited and generally inferior to
Vision-Transformers-based and Vision-Mamba-based models in most comparative
tests. Future research directions for enhancing Vision-LSTM are recommended.
The source code is available from this https URL

近期自回归网络的线性复杂度提升极大地推动了研究进展，在大型语言模型上表现出色。代表性模型为扩展长短期记忆网络 (xLSTM)，其融合了门控机制和记忆结构，在长序列语言任务上与 Transformer 架构可媲美。自回归网络如 xLSTM 可利用图像串行化扩展其应用于诸如分类和分割等视觉任务。尽管现有研究已经在图像分类方面证明了 Vision-LSTM 的出色结果，但其在图像语义分割方面的表现尚未得到验证。我们的研究首次尝试评估 Vision-LSTM 在遥感图像语义分割中的有效性。此评估基于一种名为 Seg-LSTM 的经过专门设计的编码器 - 解码器架构，并与目前最先进的分割网络进行比较。我们的研究发现 Vision-LSTM 在语义分割方面的表现受限，一般较 Vision-Transformers 和 Vision-Mamba 模型差强人意。建议未来研究方向是增强 Vision-LSTM。源代码可从此链接获取。

Seg-LSTM: 遥感图像语义分割的 xLSTM 性能

Seg-LSTM: Performance of xLSTM for Semantic Segmentation of Remotely  Sensed Images

Transformers are widely used as generic backbones in computer vision, despite
initially introduced for natural language processing. Recently, the Long
Short-Term Memory (LSTM) has been extended to a scalable and performant
architecture - the xLSTM - which overcomes long-standing LSTM limitations via
exponential gating and parallelizable matrix memory structure. In this report,
we introduce Vision-LSTM (ViL), an adaption of the xLSTM building blocks to
computer vision. ViL comprises a stack of xLSTM blocks where odd blocks process
the sequence of patch tokens from top to bottom while even blocks go from
bottom to top. Experiments show that ViL holds promise to be further deployed
as new generic backbone for computer vision architectures.

将 xLSTM 构建模块应用于计算机视觉领域，ViL 展现出成为新的通用骨干框架的潜力。