Transformer-based segmentation methods face the challenge of efficient
inference when dealing with high-resolution images. Recently, several linear
attention architectures, such as Mamba and RWKV, have attracted much attention
as they can process long sequences efficiently. In this work, we focus on
designing an efficient segment-anything model by exploring these different
architectures. Specifically, we design a mixed backbone that contains
convolution and RWKV operation, which achieves the best for both accuracy and
efficiency. In addition, we design an efficient decoder to utilize the
multiscale tokens to obtain high-quality masks. We denote our method as
RWKV-SAM, a simple, effective, fast baseline for SAM-like models. Moreover, we
build a benchmark containing various high-quality segmentation datasets and
jointly train one efficient yet high-quality segmentation model using this
benchmark. Based on the benchmark results, our RWKV-SAM achieves outstanding
performance in efficiency and segmentation quality compared to transformers and
other linear attention models. For example, compared with the same-scale
transformer model, RWKV-SAM achieves more than 2x speedup and can achieve
better segmentation performance on various datasets. In addition, RWKV-SAM
outperforms recent vision Mamba models with better classification and semantic
segmentation results. Code and models will be publicly available.

通过探索不同的线性注意力架构，设计了一种高效的分割模型 RWKV-SAM，具有最佳的精度和效率。同时，利用多尺度令牌设计了一个高质量的解码器来获取高质量的蒙版，并在各种高质量的分割数据集上训练了一个高效且高质量的模型。在效率和分割质量方面，RWKV-SAM 相对于 Transformer 和其他线性注意力模型表现出色，并超过了最近的视觉模型 Mamba。