Despite their popularity across a wide range of domains, regression neural networks are prone to overfitting complex datasets. In this work, we propose a loss function termed Random Linear Projections (RLP) loss, which is empirically shown to mitigate overfitting. With RLP loss, the distance between sets of hyperplanes connecting fixed-size subsets of the neural network's feature-prediction pairs and feature-label pairs is minimized. The intuition behind this loss derives from the notion that if two functions share the same hyperplanes connecting all subsets of feature-label pairs, then these functions must necessarily be equivalent. Our empirical studies, conducted across benchmark datasets and representative synthetic examples, demonstrate the improvements of the proposed RLP loss over mean squared error (MSE). Specifically, neural networks trained with the RLP loss achieve better performance while requiring fewer data samples and are more robust to additive noise. We provide theoretical analysis supporting our empirical findings.

我们提出了一种名为随机线性投影（RLP）损失的损失函数，通过该损失函数可以减轻复杂数据集上过拟合的问题。经过在基准数据集和典型合成例子上的实证研究，我们发现使用RLP损失训练的神经网络在性能方面取得了改进，且需要更少的数据样本，对加性噪声更加鲁棒，同时我们提供了支持我们实证结果的理论分析。

基于超平面的回归神经网络的随机线性投影损失