Flatness of the loss curve is conjectured to be connected to the
generalization ability of machine learning models, in particular neural
networks. While it has been empirically observed that flatness measures
consistently correlate strongly with generalization, it is still an open
theoretical problem why and under which circumstances flatness is connected to
generalization, in particular in light of reparameterizations that change
certain flatness measures but leave generalization unchanged. We investigate
the connection between flatness and generalization by relating it to the
interpolation from representative data, deriving notions of representativeness,
and feature robustness. The notions allow us to rigorously connect flatness and
generalization and to identify conditions under which the connection holds.
Moreover, they give rise to a novel, but natural relative flatness measure that
correlates strongly with generalization, simplifies to ridge regression for
ordinary least squares, and solves the reparameterization issue.

本文研究了损失曲线的平直性与机器学习模型的泛化能力之间的联系，通过推导出代表性数据的概念，并提出新的相对平直性的度量方法，解决了薄板回归等复杂度量方法所带来的重新参数化的问题。

相对平坦性和泛化

Relative Flatness and Generalization

Deep learning has been widely accepted as a promising solution for medical
image segmentation, given a sufficiently large representative dataset of images
with corresponding annotations. With ever increasing amounts of annotated
medical datasets, it is infeasible to train a learning method always with all
data from scratch. This is also doomed to hit computational limits, e.g.,
memory or runtime feasible for training. Incremental learning can be a
potential solution, where new information (images or anatomy) is introduced
iteratively. Nevertheless, for the preservation of the collective information,
it is essential to keep some "important" (i.e. representative) images and
annotations from the past, while adding new information. In this paper, we
introduce a framework for applying incremental learning for segmentation and
propose novel methods for selecting representative data therein. We
comparatively evaluate our methods in different scenarios using MR images and
validate the increased learning capacity with using our methods.

本文提出了一种应用于医学图像分割的增量学习框架，并提出了用于选择代表性数据的新方法，使用 MR 图像在不同情景下进行了比较评估，证实了使用该方法可以增加学习能力。