We introduce a new local sparse attention layer that preserves two-dimensional geometry and locality. We show that by just replacing the dense attention layer of SAGAN with our construction, we obtain very significant FID, Inception score and pure visual improvements. FID score is improved from $18.65$ to $15.94$ on ImageNet, keeping all other parameters the same. The sparse attention patterns that we propose for our new layer are designed using a novel information theoretic criterion that uses information flow graphs. We also present a novel way to invert Generative Adversarial Networks with attention. Our method extracts from the attention layer of the discriminator a saliency map, which we use to construct a new loss function for the inversion. This allows us to visualize the newly introduced attention heads and show that they indeed capture interesting aspects of two-dimensional geometry of real images.

本文提出了一种新的局部稀疏注意层，通过使用信息流图的新颖信息论准则来设计我们提出的稀疏注意模式，将 SAGAN 中密集的注意层替换为我们的构造，可获得非常显著的 FID、Inception 分数和纯视觉改进。同时，还提出了对注意机制进行反演的方法，并通过生成的显著性图可视化了所引入的注意头，展示他们的确捕捉到了真实图像的二维几何有趣方面。

本地 GAN：为生成模型设计二维本地注意机制