Coping with distributional shifts is an important part of transfer learning
methods in order to perform well in real-life tasks. However, most of the
existing approaches in this area either focus on an ideal scenario in which the
data does not contain noises or employ a complicated training paradigm or model
design to deal with distributional shifts. In this paper, we revisit the
robustness of the minimum error entropy (MEE) criterion, a widely used
objective in statistical signal processing to deal with non-Gaussian noises,
and investigate its feasibility and usefulness in real-life transfer learning
regression tasks, where distributional shifts are common. Specifically, we put
forward a new theoretical result showing the robustness of MEE against
covariate shift. We also show that by simply replacing the mean squared error
(MSE) loss with the MEE on basic transfer learning algorithms such as
fine-tuning and linear probing, we can achieve competitive performance with
respect to state-of-the-art transfer learning algorithms. We justify our
arguments on both synthetic data and 5 real-world time-series data.

本文旨在探讨最小误差熵准则 (MEE) 在非高斯噪声情况下的鲁棒性，并将其运用在实际的迁移学习回归任务中，结果表明仅通过将 MSE 损失替换为 MEE，我们就能够在 fine-tuning 和线性探测等基本的迁移学习算法上取得与最先进的算法竞争性的表现。

最小误差熵准则的鲁棒性再探：一项迁移学习案例研究

Revisiting the Robustness of the Minimum Error Entropy Criterion: A  Transfer Learning Case Study

We consider the minimum error entropy (MEE) criterion and an empirical risk
minimization learning algorithm in a regression setting. A learning theory
approach is presented for this MEE algorithm and explicit error bounds are
provided in terms of the approximation ability and capacity of the involved
hypothesis space when the MEE scaling parameter is large. Novel asymptotic
analysis is conducted for the generalization error associated with Renyi's
entropy and a Parzen window function, to overcome technical difficulties arisen
from the essential differences between the classical least squares problems and
the MEE setting. A semi-norm and the involved symmetrized least squares error
are introduced, which is related to some ranking algorithms.

研究了最小误差熵准则和经验风险最小化算法在回归设置中的应用，提出了一种学习理论方法，并在 MEE 缩放参数较大的情况下提供了显式误差界，在泛化误差方面进行了新颖的渐近分析，并介绍了半范数和涉及的对称最小二乘误差，与一些排名算法有关。