In this paper, we study Multiscale Vision Transformers (MViTv2) as a unified
architecture for image and video classification, as well as object detection.
We present an improved version of MViT that incorporates decomposed relative
positional embeddings and residual pooling connections. We instantiate this
architecture in five sizes and evaluate it for ImageNet classification, COCO
detection and Kinetics video recognition where it outperforms prior work. We
further compare MViTv2s' pooling attention to window attention mechanisms where
it outperforms the latter in accuracy/compute. Without bells-and-whistles,
MViTv2 has state-of-the-art performance in 3 domains: 88.8% accuracy on
ImageNet classification, 58.7 boxAP on COCO object detection as well as 86.1%
on Kinetics-400 video classification. Code and models are available at
this https URL

本文探究了多尺度视觉变换器 (MViTv2) 作为统一的图像和视频分类以及物体检测的架构，提出了一种改进版本的架构，将分解相对位置嵌入和残差池连接融入 MViTv2，并应用在 ImageNet 分类、COCO 检测和 Kinetics 视频识别中，取得了优异的性能，在三个领域的实验表明，相比于传统的拼合注意力机制，MViTv2 的池化类型的注意力机制可以更好地进行特征提取和信息编码。

MViTv2: 改进的多尺度视觉 Transformer 用于分类和检测

MViTv2: Improved Multiscale Vision Transformers for Classification and  Detection

We present Multiscale Vision Transformers (MViT) for video and image
recognition, by connecting the seminal idea of multiscale feature hierarchies
with transformer models. Multiscale Transformers have several
channel-resolution scale stages. Starting from the input resolution and a small
channel dimension, the stages hierarchically expand the channel capacity while
reducing the spatial resolution. This creates a multiscale pyramid of features
with early layers operating at high spatial resolution to model simple
low-level visual information, and deeper layers at spatially coarse, but
complex, high-dimensional features. We evaluate this fundamental architectural
prior for modeling the dense nature of visual signals for a variety of video
recognition tasks where it outperforms concurrent vision transformers that rely
on large scale external pre-training and are 5-10x more costly in computation
and parameters. We further remove the temporal dimension and apply our model
for image classification where it outperforms prior work on vision
transformers. Code is available at:
this https URL

Multiscale Vision Transformers 是一种用于视频和图像识别的多尺度特征层级转换器，它可基于视觉信号的密集性建模逐渐展开通道容量和降低空间分辨率得到的多尺度特征金字塔，它能优于大规模的外部预训练及在计算和参数方面更为昂贵的传统视觉转换器。