Heteroscedastic regression is the task of supervised learning where each label is subject to noise from a different distribution. This noise can be caused by the labelling process, and impacts negatively the performance of the learning algorithm as it violates the i.i.d. assumptions. In many situations however, the labelling process is able to estimate the variance of such distribution for each label, which can be used as an additional information to mitigate this impact. We adapt an inverse-variance weighted mean square error, based on the Gauss-Markov theorem, for parameter optimization on neural networks. We introduce Batch Inverse-Variance, a loss function which is robust to near-ground truth samples, and allows to control the effective learning rate. Our experimental results show that BIV improves significantly the performance of the networks on two noisy datasets, compared to L2 loss, inverse-variance weighting, as well as a filtering-based baseline.

本文介绍了一种基于逆方差权重的均方误差适应于神经网络的参数优化方法，同时提出了Batch Inverse-Variance（BIV）作为一种损失函数用来控制学习率，并展示了BIV相比L2 loss、逆方差加权和过滤式基准方法在两个噪声数据集上显著提高了神经网络的性能。

批次逆变权重：深度异方差回归