We present a detailed study of estimation errors in terms of surrogate loss
estimation errors. We refer to such guarantees as $\mathscr{H}$-consistency
estimation error bounds, since they account for the hypothesis set
$\mathscr{H}$ adopted. These guarantees are significantly stronger than
$\mathscr{H}$-calibration or $\mathscr{H}$-consistency. They are also more
informative than similar excess error bounds derived in the literature, when
$\mathscr{H}$ is the family of all measurable functions. We prove general
theorems providing such guarantees, for both the distribution-dependent and
distribution-independent settings. We show that our bounds are tight, modulo a
convexity assumption. We also show that previous excess error bounds can be
recovered as special cases of our general results.
We then present a series of explicit bounds in the case of the zero-one loss,
with multiple choices of the surrogate loss and for both the family of linear
functions and neural networks with one hidden-layer. We further prove more
favorable distribution-dependent guarantees in that case. We also present a
series of explicit bounds in the case of the adversarial loss, with surrogate
losses based on the supremum of the $\rho$-margin, hinge or sigmoid loss and
for the same two general hypothesis sets. Here too, we prove several
enhancements of these guarantees under natural distributional assumptions.
Finally, we report the results of simulations illustrating our bounds and their
tightness.

本文主要研究代理损失估计误差及其保证方法，提供零一损失和对抗性损失两种情况下的具体保证方法，并通过模拟实验证明了其紧密性。

$\mathscr {H}$- 替代损失函数最小化器的一致性估计误差

$\mathscr{H}$-Consistency Estimation Error of Surrogate Loss Minimizers

Differentiable Architecture Search (DARTS) is now a widely disseminated
weight-sharing neural architecture search method. However, it suffers from
well-known performance collapse due to an inevitable aggregation of skip
connections. In this paper, we first disclose that its root cause lies in an
unfair advantage in exclusive competition. Through experiments, we show that if
either of two conditions is broken, the collapse disappears. Thereby, we
present a novel approach called Fair DARTS where the exclusive competition is
relaxed to be collaborative. Specifically, we let each operation's
architectural weight be independent of others. Yet there is still an important
issue of discretization discrepancy. We then propose a zero-one loss to push
architectural weights towards zero or one, which approximates an expected
multi-hot solution. Our experiments are performed on two mainstream search
spaces, and we derive new state-of-the-art results on CIFAR-10 and ImageNet.
Our code is available on this https URL .

通过实验，揭示了 Differentiable Architecture Search 权重共享神经结构搜索的性能下降是由于竞争中的不公平优势导致的，并提出了 “公平 DARTS” 方法，在其中将独占竞争放松为合作。实施了零 - 一损失来推动对架构权重的期望约束，并获得了 CIFAR-10 和 ImageNet 的最新技术结果。

公平 DARTS：消除可微架构搜索中的不公平优势

Fair DARTS: Eliminating Unfair Advantages in Differentiable Architecture  Search

Uncertainty sampling, a popular active learning algorithm, is used to reduce
the amount of data required to learn a classifier, but it has been observed in
practice to converge to different parameters depending on the initialization
and sometimes to even better parameters than standard training on all the data.
In this work, we give a theoretical explanation of this phenomenon, showing
that uncertainty sampling on a convex loss can be interpreted as performing a
preconditioned stochastic gradient step on a smoothed version of the population
zero-one loss that converges to the population zero-one loss. Furthermore,
uncertainty sampling moves in a descent direction and converges to stationary
points of the smoothed population zero-one loss. Experiments on synthetic and
real datasets support this connection.

本文通过理论分析和实验证明了，基于凸损失上的不确定性采样算法可以被解释为在平滑版本的无约束损失函数上进行预处理的随机梯度下降步骤，且可以收敛到该函数的极小值点。

利用不确定性采样的 0-1 损失预条件随机梯度下降

Uncertainty Sampling is Preconditioned Stochastic Gradient Descent on  Zero-One Loss

A sequence of random variables is exchangeable if its joint distribution is
invariant under variable permutations. We introduce exchangeable variable
models (EVMs) as a novel class of probabilistic models whose basic building
blocks are partially exchangeable sequences, a generalization of exchangeable
sequences. We prove that a family of tractable EVMs is optimal under zero-one
loss for a large class of functions, including parity and threshold functions,
and strictly subsumes existing tractable independence-based model families.
Extensive experiments show that EVMs outperform state of the art classifiers
such as SVMs and probabilistic models which are solely based on independence
assumptions.

引入了可交换变量模型 (EVMs) 作为一类新型的概率模型，证明了一族可处理的 EVMs 在零一损失下对于包括奇偶性和阈值函数等大类函数是最优的，并严格包括现有的独立性模型家族，实验证明 EVMs 比仅基于独立性假设的状态艺术分类器如 SVM 和概率模型表现更好。