In contrast to fully connected networks, Convolutional Neural Networks (CNNs)
achieve efficiency by learning weights associated with local filters with a
finite spatial extent. An implication of this is that a filter may know what it
is looking at, but not where it is positioned in the image. In this paper, we
first test this hypothesis and reveal that a surprising degree of absolute
position information is encoded in commonly used CNNs. We show that zero
padding drives CNNs to encode position information in their internal
representations, while a lack of padding precludes position encoding. This
gives rise to deeper questions about the role of position information in CNNs:
(i) What boundary heuristics enable optimal position encoding for downstream
tasks?; (ii) Does position encoding affect the learning of semantic
representations?; (iii) Does position encoding always improve performance? To
provide answers, we perform the largest case study to date on the role that
padding and border heuristics play in CNNs. We design novel tasks which allow
us to quantify boundary effects as a function of the distance to the border.
Numerous semantic objectives reveal the effect of the border on semantic
representations. Finally, we demonstrate the implications of these findings on
multiple real-world tasks to show that position information can both help or
hurt performance.

本论文研究了卷积神经网络（CNNs）中位置信息的编码方式和其对语义表征的影响，通过设计新颖的任务和语义目标测试终止边界对语义表征的影响，揭示零填充驱动 CNNs 在其内部表征中进行位置信息编码的新发现，并证明位置信息既可以帮助也可以阻碍性能。