There has long been plenty of theoretical and empirical evidence supporting
the success of ensemble learning. Deep ensembles in particular take advantage
of training randomness and expressivity of individual neural networks to gain
prediction diversity, ultimately leading to better generalization, robustness
and uncertainty estimation. In respect of generalization, it is found that
pursuing wider local minima result in models being more robust to shifts
between training and testing sets. A natural research question arises out of
these two approaches as to whether a boost in generalization ability can be
achieved if ensemble learning and loss sharpness minimization are integrated.
Our work investigates this connection and proposes DASH - a learning algorithm
that promotes diversity and flatness within deep ensembles. More concretely,
DASH encourages base learners to move divergently towards low-loss regions of
minimal sharpness. We provide a theoretical backbone for our method along with
extensive empirical evidence demonstrating an improvement in ensemble
generalizability.

我们的工作调查了集成学习和损失尖锐度最小化相结合是否能增加泛化能力，并提出了 DASH 算法，该算法在深度集成中促进多样性和扁平性，理论和实证证据表明集成的可泛化性得到了改进。

多样性感知的无偏锐度最小化集成方法

Diversity-Aware Agnostic Ensemble of Sharpness Minimizers

Neural structure learning is of paramount importance for scientific discovery
and interpretability. Yet, contemporary pruning algorithms that focus on
computational resource efficiency face algorithmic barriers to select a
meaningful model that aligns with domain expertise. To mitigate this challenge,
we propose DASH, which guides pruning by available domain-specific structural
information. In the context of learning dynamic gene regulatory network models,
we show that DASH combined with existing general knowledge on interaction
partners provides data-specific insights aligned with biology. For this task,
we show on synthetic data with ground truth information and two real world
applications the effectiveness of DASH, which outperforms competing methods by
a large margin and provides more meaningful biological insights. Our work shows
that domain specific structural information bears the potential to improve
model-derived scientific insights.

通过使用特定领域的结构信息，我们提出了 DASH 修剪算法，并在学习动态基因调控网络模型的任务中展示了其在提供与生物学相关的数据洞察力方面的效果优于竞争方法。

不是所有的票务都是平等的，我们明白：借助领域特定知识指导剪枝

Not all tickets are equal and we know it: Guiding pruning with  domain-specific knowledge

While neural architecture search (NAS) has enabled automated machine learning
(AutoML) for well-researched areas, its application to tasks beyond computer
vision is still under-explored. As less-studied domains are precisely those
where we expect AutoML to have the greatest impact, in this work we study NAS
for efficiently solving diverse problems. Seeking an approach that is fast,
simple, and broadly applicable, we fix a standard convolutional network (CNN)
topology and propose to search for the right kernel sizes and dilations its
operations should take on. This dramatically expands the model's capacity to
extract features at multiple resolutions for different types of data while only
requiring search over the operation space. To overcome the efficiency
challenges of naive weight-sharing in this search space, we introduce DASH, a
differentiable NAS algorithm that computes the mixture-of-operations using the
Fourier diagonalization of convolution, achieving both a better asymptotic
complexity and an up-to-10x search time speedup in practice. We evaluate DASH
on ten tasks spanning a variety of application domains such as PDE solving,
protein folding, and heart disease detection. DASH outperforms state-of-the-art
AutoML methods in aggregate, attaining the best-known automated performance on
seven tasks. Meanwhile, on six of the ten tasks, the combined search and
retraining time is less than 2x slower than simply training a CNN backbone that
is far less accurate.

本文研究了基于神经架构搜索（NAS）的自动机器学习（AutoML）在计算机视觉以外的任务上的应用，作者提出了一种名为 DASH 的不同 iable NAS 算法，用于搜索卷积神经网络（CNN）的卷积核，实现了在多种应用领域上最先进的自动化性能。