This paper examines the relationship between the behavior of a neural network
and the distribution formed from the projections of the data records into the
space spanned by the low-order principal components of the training data. For
example, in a benchmark calculation involving rotated and unrotated MNIST
digits, classes (digits) that are mapped far from the origin in a
low-dimensional principal component space and that overlap minimally with other
digits converge rapidly and exhibit high degrees of accuracy in neural network
calculations that employ the associated components of each data record as
inputs. Further, if the space spanned by these low-order principal components
is divided into bins and the input data records that are mapped into a given
bin averaged, the resulting pattern can be distinguished by its geometric
features which interpolate between those of adjacent bins in an analogous
manner to variational autoencoders. Based on this observation, a simply
realized data balancing procedure can be realized by evaluating the entropy
associated with each histogram bin and subsequently repeating the original
image data associated with the bin by a number of times that is determined from
this entropy.

本文研究了神经网络行为与数据记录在训练数据的低阶主成分所构成的分布之间的关系，通过评估每个直方图箱子相关的熵来实现简单的数据平衡程序。