Spatial redundancy widely exists in visual recognition tasks, i.e.,
discriminative features in an image or video frame usually correspond to only a
subset of pixels, while the remaining regions are irrelevant to the task at
hand. Therefore, static models which process all the pixels with an equal
amount of computation result in considerable redundancy in terms of time and
space consumption. In this paper, we formulate the image recognition problem as
a sequential coarse-to-fine feature learning process, mimicking the human
visual system. Specifically, the proposed Glance and Focus Network (GFNet)
first extracts a quick global representation of the input image at a low
resolution scale, and then strategically attends to a series of salient (small)
regions to learn finer features. The sequential process naturally facilitates
adaptive inference at test time, as it can be terminated once the model is
sufficiently confident about its prediction, avoiding further redundant
computation. It is worth noting that the problem of locating discriminant
regions in our model is formulated as a reinforcement learning task, thus
requiring no additional manual annotations other than classification labels.
GFNet is general and flexible as it is compatible with any off-the-shelf
backbone models (such as MobileNets, EfficientNets and TSM), which can be
conveniently deployed as the feature extractor. Extensive experiments on a
variety of image classification and video recognition tasks and with various
backbone models demonstrate the remarkable efficiency of our method. For
example, it reduces the average latency of the highly efficient MobileNet-V3 on
an iPhone XS Max by 1.3x without sacrificing accuracy. Code and pre-trained
models are available at this https URL.

本文提出了一个新的序列特征学习方法，Glance and Focus Network（GFNet），用于图像识别问题中的减少空间冗余和时间复杂度的优化，GFNet 采用类似人类视觉系统的粗到细的学习方式处理图像，并通过强化学习的方式定位图像中的显著区域，从而避免了手动标注的需求。实验表明，GFNet 能够大幅度降低 MobileNet-V3 在 iPhone XS Max 上的平均延迟 (1.3x)，而精度没有任何损失。