Convolutional layers are a fundamental component of most image-related
models. These layers often implement by default a static padding policy (\eg
zero padding), to control the scale of the internal representations, and to
allow kernel activations centered on the border regions. In this work we
identify Padding Aware Neurons (PANs), a type of filter that is found in most
(if not all) convolutional models trained with static padding. PANs focus on
the characterization and recognition of input border location, introducing a
spatial inductive bias into the model (e.g., how close to the input's border a
pattern typically is). We propose a method to identify PANs through their
activations, and explore their presence in several popular pre-trained models,
finding PANs on all models explored, from dozens to hundreds. We discuss and
illustrate different types of PANs, their kernels and behaviour. To understand
their relevance, we test their impact on model performance, and find padding
and PANs to induce strong and characteristic biases in the data. Finally, we
discuss whether or not PANs are desirable, as well as the potential side
effects of their presence in the context of model performance, generalisation,
efficiency and safety.

通过激活值识别和研究静态填充卷积模型中的边界感知神经元，探索它们对模型性能的影响以及其在数据中引起的偏差。

填充感知神经元

Padding Aware Neurons

SinGAN shows impressive capability in learning internal patch distribution
despite its limited effective receptive field. We are interested in knowing how
such a translation-invariant convolutional generator could capture the global
structure with just a spatially i.i.d. input. In this work, taking SinGAN and
StyleGAN2 as examples, we show that such capability, to a large extent, is
brought by the implicit positional encoding when using zero padding in the
generators. Such positional encoding is indispensable for generating images
with high fidelity. The same phenomenon is observed in other generative
architectures such as DCGAN and PGGAN. We further show that zero padding leads
to an unbalanced spatial bias with a vague relation between locations. To offer
a better spatial inductive bias, we investigate alternative positional
encodings and analyze their effects. Based on a more flexible positional
encoding explicitly, we propose a new multi-scale training strategy and
demonstrate its effectiveness in the state-of-the-art unconditional generator
StyleGAN2. Besides, the explicit spatial inductive bias substantially improve
SinGAN for more versatile image manipulation.

该论文研究了产生式模型的位置编码对于生成高保真度图像的作用，发现使用零填充时的隐式位置编码是生成器捕捉全局结构的关键，并且使用一种更灵活的显式位置编码可以显著提高产生式模型的性能。