Vision Transformers (ViTs) have attracted a lot of popularity in recent
years, due to their exceptional capabilities in modeling long-range spatial
dependencies and scalability for large scale training. Although the training
parallelism of self-attention mechanism plays an important role in retaining
great performance, its quadratic complexity baffles the application of ViTs in
many scenarios which demand fast inference. This effect is even more pronounced
in applications in which autoregressive modeling of input features is required.
In Natural Language Processing (NLP), a new stream of efforts have proposed
parallelizable models with recurrent formulation that allows for efficient
inference in generative applications. Inspired by this trend, we propose a new
class of computer vision models, dubbed Vision Retention Networks (ViR), with
dual parallel and recurrent formulations, which strike an optimal balance
between fast inference and parallel training with competitive performance. In
particular, ViR scales favorably for image throughput and memory consumption in
tasks that require higher-resolution images due to its flexible formulation in
processing large sequence lengths. The ViR is the first attempt to realize dual
parallel and recurrent equivalency in a general vision backbone for recognition
tasks. We have validated the effectiveness of ViR through extensive experiments
with different dataset sizes and various image resolutions and achieved
competitive performance. Our code and pretrained models will be made publicly
available.

基于自注意力机制的视觉转换器（ViTs）因其对长距离空间依赖关系和大规模训练的出色能力而受到了广泛的关注，然而，其二次复杂度限制了其在许多需要快速推理的场景中的应用。受到自然语言处理中并行化模型的启发，我们提出了一种新的计算机视觉模型，命名为视觉保留网络（ViR），具有双并行和循环结构，以在快速推理和并行训练之间达到最佳平衡，并在需要更高分辨率图像的任务中具有可扩展性。ViR 是第一个尝试在通用视觉骨干网络中实现双并行和循环等效性的方法，我们通过大量实验证实了 ViR 的有效性，并提供了代码和预训练模型的公开获取。