The Segment Anything Model (SAM) has significantly advanced interactive
segmentation but struggles with high-resolution images crucial for
high-precision segmentation. This is primarily due to the quadratic space
complexity of SAM-implemented attention and the length extrapolation issue in
common global attention. This study proposes HRSAM that integrates Flash
Attention and incorporates Plain, Shifted and newly proposed Cycle-scan Window
(PSCWin) attention to address these issues. The shifted window attention is
redesigned with padding to maintain consistent window sizes, enabling effective
length extrapolation. The cycle-scan window attention adopts the recently
developed State Space Models (SSMs) to ensure global information exchange with
minimal computational overhead. Such window-based attention allows HRSAM to
perform effective attention computations on scaled input images while
maintaining low latency. Moreover, we further propose HRSAM++ that additionally
employs a multi-scale strategy to enhance HRSAM's performance. The experiments
on the high-precision segmentation datasets HQSeg44K and DAVIS show that
high-resolution inputs enable the SAM-distilled HRSAM models to outperform the
teacher model while maintaining lower latency. Compared to the SOTAs, HRSAM
achieves a 1.56 improvement in interactive segmentation's NoC95 metric with
only 31% of the latency. HRSAM++ further enhances the performance, achieving a
1.63 improvement in NoC95 with just 38% of the latency.

本研究提出了一种名为 HRSAM 的模型，通过整合 Flash Attention 和 Plain、Shifted 以及新提出的 Cycle-scan Window (PSCWin) attention 解决了高分辨率图像的高精度分割中的一些问题，并且进一步提出了 HRSAM++，通过采用多尺度策略进一步提升性能。实验结果表明，相比于现有方法，HRSAM 和 HRSAM++ 在交互式分割中取得了较好的效果，具有更低的延迟。