Neural networks are not learning optimal decision boundaries. We show that
decision boundaries are situated in areas of low training data density. They
are impacted by few training samples which can easily lead to overfitting. We
provide a simple algorithm performing a weighted average of the prediction of a
sample and its nearest neighbors' (computed in latent space) leading to a minor
favorable outcomes for a variety of important measures for neural networks. In
our evaluation, we employ various self-trained and pre-trained convolutional
neural networks to show that our approach improves (i) resistance to label
noise, (ii) robustness against adversarial attacks, (iii) classification
accuracy, and to some degree even (iv) interpretability. While improvements are
not necessarily large in all four areas, our approach is conceptually simple,
i.e., improvements come without any modification to network architecture,
training procedure or dataset. Furthermore, they are in stark contrast to prior
works that often require trade-offs among the four objectives or provide
valuable, but non-actionable insights.

神经网络训练中，决策边界位于训练数据密度低的区域，受到少数训练样本的影响容易导致过度拟合；我们提供了一种简单的算法，通过对样本及其最近邻的预测（在潜在空间中计算）进行加权平均，以优化神经网络在标签噪声、对抗攻击、分类准确性等方面的性能。