The Manual labeling of data is and will remain a costly endeavor. For this
reason, semi-supervised learning remains a topic of practical importance. The
recently proposed Ladder Network is one such approach that has proven to be
very successful. In addition to the supervised objective, the Ladder Network
also adds an unsupervised objective corresponding to the reconstruction costs
of a stack of denoising autoencoders. Although the empirical results are
impressive, the Ladder Network has many components intertwined, whose
contributions are not obvious in such a complex architecture. In order to help
elucidate and disentangle the different ingredients in the Ladder Network
recipe, this paper presents an extensive experimental investigation of variants
of the Ladder Network in which we replace or remove individual components to
gain more insight into their relative importance. We find that all of the
components are necessary for achieving optimal performance, but they do not
contribute equally. For semi-supervised tasks, we conclude that the most
important contribution is made by the lateral connection, followed by the
application of noise, and finally the choice of what we refer to as the
`combinator function' in the decoder path. We also find that as the number of
labeled training examples increases, the lateral connections and reconstruction
criterion become less important, with most of the improvement in generalization
being due to the injection of noise in each layer. Furthermore, we present a
new type of combinator function that outperforms the original design in both
fully- and semi-supervised tasks, reducing record test error rates on
Permutation-Invariant MNIST to 0.57% for the supervised setting, and to 0.97%
and 1.0% for semi-supervised settings with 1000 and 100 labeled examples
respectively.

本文研究了一种神经网络架构 Ladder Network，通过剔除或替换部分组件，探究了其各个组成部分的贡献及重要性，并提出了新型组合函数，在全监督和半监督任务中均取得了较好的性能表现。