There has long been plenty of theoretical and empirical evidence supporting
the success of ensemble learning. Deep ensembles in particular take advantage
of training randomness and expressivity of individual neural networks to gain
prediction diversity, ultimately leading to better generalization, robustness
and uncertainty estimation. In respect of generalization, it is found that
pursuing wider local minima result in models being more robust to shifts
between training and testing sets. A natural research question arises out of
these two approaches as to whether a boost in generalization ability can be
achieved if ensemble learning and loss sharpness minimization are integrated.
Our work investigates this connection and proposes DASH - a learning algorithm
that promotes diversity and flatness within deep ensembles. More concretely,
DASH encourages base learners to move divergently towards low-loss regions of
minimal sharpness. We provide a theoretical backbone for our method along with
extensive empirical evidence demonstrating an improvement in ensemble
generalizability.

我们的工作调查了集成学习和损失尖锐度最小化相结合是否能增加泛化能力，并提出了 DASH 算法，该算法在深度集成中促进多样性和扁平性，理论和实证证据表明集成的可泛化性得到了改进。