Suitable lateral connections between encoder and decoder are shown to allow
higher layers of a denoising autoencoder (dAE) to focus on invariant
representations. In regular autoencoders, detailed information needs to be
carried through the highest layers but lateral connections from encoder to
decoder relieve this pressure. It is shown that abstract invariant features can
be translated to detailed reconstructions when invariant features are allowed
to modulate the strength of the lateral connection. Three dAE structures with
modulated and additive lateral connections, and without lateral connections
were compared in experiments using real-world images. The experiments verify
that adding modulated lateral connections to the model 1) improves the accuracy
of the probability model for inputs, as measured by denoising performance; 2)
results in representations whose degree of invariance grows faster towards the
higher layers; and 3) supports the formation of diverse invariant poolings.

研究表明，编码器和解码器之间的适当侧向连接可以使去噪自编码器（dAE）的更高层专注于不变表示。添加被允许调制侧向连接的不变特征可以将抽象不变特征翻译为详细的重构，并支持形成不同的不变池。通过在真实世界图像上进行实验，发现添加调制的侧向连接到模型中可以提高模型对输入的概率模型准确性，产生的表示不变度在更高层次上增加得更快，并支持形成多样的不变池塘。