Flow-based generative models have shown an excellent ability to explicitly
learn the probability density function of data via a sequence of invertible
transformations. Yet, learning attentions in generative flows remains
understudied, while it has made breakthroughs in other domains. To fill the
gap, this paper introduces two types of invertible attention mechanisms, i.e.,
map-based and transformer-based attentions, for both unconditional and
conditional generative flows. The key idea is to exploit a masked scheme of
these two attentions to learn long-range data dependencies in the context of
generative flows. The masked scheme allows for invertible attention modules
with tractable Jacobian determinants, enabling its seamless integration at any
positions of the flow-based models. The proposed attention mechanisms lead to
more efficient generative flows, due to their capability of modeling the
long-term data dependencies. Evaluation on multiple image synthesis tasks shows
that the proposed attention flows result in efficient models and compare
favorably against the state-of-the-art unconditional and conditional generative
flows.

本文介绍了两种可逆注意力机制，即基于映射和变换器的注意力，用于无条件和有条件的生成流。这些注意力机制利用掩码方案学习生成流上的长程数据依赖关系，并产生更有效的生成流模型，证明了其与最先进的生成流相比的优越性。