The feature map obtained from the denoising autoencoder (DAE) is investigated
by determining transportation dynamics of the DAE, which is a cornerstone for
deep learning. Despite the rapid development in its application, deep neural
networks remain analytically unexplained, because the feature maps are nested
and parameters are not faithful. In this paper, we address the problem of the
formulation of nested complex of parameters by regarding the feature map as a
transport map. Even when a feature map has different dimensions between input
and output, we can regard it as a transportation map by considering that both
the input and output spaces are embedded in a common high-dimensional space. In
addition, the trajectory is a geometric object and thus, is independent of
parameterization. In this manner, transportation can be regarded as a universal
character of deep neural networks. By determining and analyzing the
transportation dynamics, we can understand the behavior of a deep neural
network. In this paper, we investigate a fundamental case of deep neural
networks: the DAE. We derive the transport map of the DAE, and reveal that the
infinitely deep DAE transports mass to decrease a certain quantity, such as
entropy, of the data distribution. These results though analytically simple,
shed light on the correspondence between deep neural networks and the
Wasserstein gradient flows.

本文通过将特征映射视为运输映射，研究了深度学习的基本案例 - 去噪自编码器（DAE）的运输动态，并揭示了无限深的 DAE 如何将 mass 输送出去以减少数据分布的某种数量，即熵，这些结果虽然在数学上很简单，但是启示我们深度神经网络和 Wasserstein 梯度流之间的对应关系。