Among existing Neural Architecture Search methods, DARTS is known for its
efficiency and simplicity. This approach applies continuous relaxation of
network representation to construct a weight-sharing supernet and enables the
identification of excellent subnets in just a few GPU days. However,
performance collapse in DARTS results in deteriorating architectures filled
with parameter-free operations and remains a great challenge to the robustness.
To resolve this problem, we reveal that the fundamental reason is the biased
estimation of the candidate importance in the search space through theoretical
and experimental analysis, and more precisely select operations via
information-based measurements. Furthermore, we demonstrate that the excessive
concern over the supernet and inefficient utilization of data in bi-level
optimization also account for suboptimal results. We adopt a more realistic
objective focusing on the performance of subnets and simplify it with the help
of the information-based measurements. Finally, we explain theoretically why
progressively shrinking the width of the supernet is necessary and reduce the
approximation error of optimal weights in DARTS. Our proposed method, named
IS-DARTS, comprehensively improves DARTS and resolves the aforementioned
problems. Extensive experiments on NAS-Bench-201 and DARTS-based search space
demonstrate the effectiveness of IS-DARTS.

对现有的神经网络架构搜索方法进行了研究，发现 DARTS 方法在寻找优秀子网络方面效率高且简单，但存在性能崩溃的问题，导致架构充斥着无需参数的操作，提出了 IS-DARTS 方法来解决这些问题，并通过理论和实验分析揭示了问题的根本原因和解决途径，验证了 IS-DARTS 的有效性。

IS-DARTS：通过精确测量候选重要性稳定 DARTS

IS-DARTS: Stabilizing DARTS through Precise Measurement on Candidate  Importance

Diffusion models have demonstrated compelling generation quality by
optimizing the variational lower bound through a simple denoising score
matching loss. In this paper, we provide theoretical evidence that the
prevailing practice of using a constant loss weight strategy in diffusion
models leads to biased estimation during the training phase. Simply optimizing
the denoising network to predict Gaussian noise with constant weighting may
hinder precise estimations of original images. To address the issue, we propose
an elegant and effective weighting strategy grounded in the theoretically
unbiased principle. Moreover, we conduct a comprehensive and systematic
exploration to dissect the inherent bias problem deriving from constant
weighting loss from the perspectives of its existence, impact and reasons.
These analyses are expected to advance our understanding and demystify the
inner workings of diffusion models. Through empirical evaluation, we
demonstrate that our proposed debiased estimation method significantly enhances
sample quality without the reliance on complex techniques, and exhibits
improved efficiency compared to the baseline method both in training and
sampling processes.

通过理论支持，本文提出一种优雅而有效的加权策略，解决了扩散模型中常数加权策略带来的估计偏差问题。并通过实证评估表明，提出的去偏估计方法在样本质量、训练效率和采样效率方面均显著优于基准方法。

去偏扩散模型的训练

Debias the Training of Diffusion Models

We consider the problem of fitting a linear model to data held by individuals
who are concerned about their privacy. Incentivizing most players to truthfully
report their data to the analyst constrains our design to mechanisms that
provide a privacy guarantee to the participants; we use differential privacy to
model individuals' privacy losses. This immediately poses a problem, as
differentially private computation of a linear model necessarily produces a
biased estimation, and existing approaches to design mechanisms to elicit data
from privacy-sensitive individuals do not generalize well to biased estimators.
We overcome this challenge through an appropriate design of the computation and
payment scheme.

研究了如何在保护隐私的前提下，通过采用不同隐私保护机制和支付方案，解决线性模型和差分隐私计算会带来估计偏差的问题。