Most hard attention models initially observe a complete scene to locate and sense informative glimpses, and predict class-label of a scene based on glimpses. However, in many applications (e.g., aerial imaging), observing an entire scene is not always feasible due to the limited time and resources available for acquisition. In this paper, we develop a Sequential Transformers Attention Model (STAM) that only partially observes a complete image and predicts informative glimpse locations solely based on past glimpses. We design our agent using DeiT-distilled and train it with a one-step actor-critic algorithm. Furthermore, to improve classification performance, we introduce a novel training objective, which enforces consistency between the class distribution predicted by a teacher model from a complete image and the class distribution predicted by our agent using glimpses. When the agent senses only 4% of the total image area, the inclusion of the proposed consistency loss in our training objective yields 3% and 8% higher accuracy on ImageNet and fMoW datasets, respectively. Moreover, our agent outperforms previous state-of-the-art by observing nearly 27% and 42% fewer pixels in glimpses on ImageNet and fMoW.

本文提出序列转换器关注模型（STAM），以在资源有限的情况下，部分观察完整图像并仅基于过去的瞥见预测信息突出位置。使用DeiT-distilled设计代理并使用一步演员-评论家算法进行培训。此外，为了提高分类性能，还介绍了一种新的培训目标，该目标强制要求完整图像中由教师模型预测的类分布与使用瞥见预测的类分布之间的一致性。在观测图像总面积仅为4％的情况下，引入一致性损失的训练目标在ImageNet和fMoW数据集上分别提高了3％和8％的准确性。此外，在ImageNet和fMoW上观察到的像素数量减少了近27％和42％。

基于一致性约束的序列Transformer关注模型在部分可观测场景中的应用