Convolutional operations have two limitations: (1) do not explicitly model
where to focus as the same filter is applied to all the positions, and (2) are
unsuitable for modeling long-range dependencies as they only operate on a small
neighborhood. While both limitations can be alleviated by attention operations,
many design choices remain to be determined to use attention, especially when
applying attention to videos. Towards a principled way of applying attention to
videos, we address the task of spatiotemporal attention cell search. We propose
a novel search space for spatiotemporal attention cells, which allows the
search algorithm to flexibly explore various design choices in the cell. The
discovered attention cells can be seamlessly inserted into existing backbone
networks, e.g., I3D or S3D, and improve video classification accuracy by more
than 2% on both Kinetics-600 and MiT datasets. The discovered attention cells
outperform non-local blocks on both datasets, and demonstrate strong
generalization across different modalities, backbones, and datasets. Inserting
our attention cells into I3D-R50 yields state-of-the-art performance on both
datasets.

提出一种新的用于搜索空间时间关注单元的方法，可以用于改善现有的骨干网络（如 I3D 或 S3D）中的视频分类精度，并在 Kinetics-600 和 MiT 数据集上超过 2％。

AttentionNAS: 基于时空注意力单元搜索的视频分类

AttentionNAS: Spatiotemporal Attention Cell Search for Video  Classification

Non-Local (NL) blocks have been widely studied in various vision tasks.
However, it has been rarely explored to embed the NL blocks in mobile neural
networks, mainly due to the following challenges: 1) NL blocks generally have
heavy computation cost which makes it difficult to be applied in applications
where computational resources are limited, and 2) it is an open problem to
discover an optimal configuration to embed NL blocks into mobile neural
networks. We propose AutoNL to overcome the above two obstacles. Firstly, we
propose a Lightweight Non-Local (LightNL) block by squeezing the transformation
operations and incorporating compact features. With the novel design choices,
the proposed LightNL block is 400x computationally cheaper} than its
conventional counterpart without sacrificing the performance. Secondly, by
relaxing the structure of the LightNL block to be differentiable during
training, we propose an efficient neural architecture search algorithm to learn
an optimal configuration of LightNL blocks in an end-to-end manner. Notably,
using only 32 GPU hours, the searched AutoNL model achieves 77.7% top-1
accuracy on ImageNet under a typical mobile setting (350M FLOPs), significantly
outperforming previous mobile models including MobileNetV2 (+5.7%), FBNet
(+2.8%) and MnasNet (+2.1%). Code and models are available at
this https URL

本文提出了 AutoNL 算法解决了 Non-Local blocks 难以嵌入移动神经网络和配置不优等问题。AutoNL 设计了轻量级的 Lightweight Non-Local block 和一种有效的神经架构搜索算法，仅使用 32 个 GPU 小时即在 ImageNet 数据集的 350M FLOPs 的移动设定下实现了 77.7％的准确率，显著优于 MobileNetV2，FBNet 和 MnasNet 等先前的移动模型。