We present Reversible Vision Transformers, a memory efficient architecture
design for visual recognition. By decoupling the GPU memory requirement from
the depth of the model, Reversible Vision Transformers enable scaling up
architectures with efficient memory usage. We adapt two popular models, namely
Vision Transformer and Multiscale Vision Transformers, to reversible variants
and benchmark extensively across both model sizes and tasks of image
classification, object detection and video classification. Reversible Vision
Transformers achieve a reduced memory footprint of up to 15.5x at roughly
identical model complexity, parameters and accuracy, demonstrating the promise
of reversible vision transformers as an efficient backbone for hardware
resource limited training regimes. Finally, we find that the additional
computational burden of recomputing activations is more than overcome for
deeper models, where throughput can increase up to 2.3x over their
non-reversible counterparts. Full code and trained models are available at
this https URL. A simpler, easy to understand and
modify version is also available at this https URL

本文介绍一种内存高效的 Reversible Vision Transformer 架构设计，将模型深度与 GPU 内存需求解耦，能够通过有效地内存使用扩展架构。在图像分类、物体检测和视频分类等多个任务中进行了广泛基准测试，表明可实现模型复杂度、参数和准确性基本相同的情况下，内存占用减少了多达 15.5 倍，Reversible Vision Transformers 是硬件资源有限培训方案的高效支撑，为更深的模型提供了更快的吞吐量。