In this work, we introduce a novel approach to regularization in
multivariable regression problems. Our regularizer, called DLoss, penalises
differences between the model's derivatives and derivatives of the data
generating function as estimated from the training data. We call these
estimated derivatives data derivatives. The goal of our method is to align the
model to the data, not only in terms of target values but also in terms of the
derivatives involved. To estimate data derivatives, we select (from the
training data) 2-tuples of input-value pairs, using either nearest neighbour or
random, selection. On synthetic and real datasets, we evaluate the
effectiveness of adding DLoss, with different weights, to the standard mean
squared error loss. The experimental results show that with DLoss (using
nearest neighbour selection) we obtain, on average, the best rank with respect
to MSE on validation data sets, compared to no regularization, L2
regularization, and Dropout.

我们介绍了一种新颖的正则化方法，称为 DLoss，在多变量回归问题中。我们的目标是通过惩罚模型的导数与从训练数据估计得出的数据生成函数的导数之间的差异，将模型与数据对齐，不仅在目标值方面，还包括相关的导数。实验证明，在使用 DLoss（使用最近邻选择）时，我们平均获得与验证数据集中的均方误差最佳排名，相比于无正则化、L2 正则化和 Dropout。