We present a detailed study of $H$-consistency bounds for regression. We
first present new theorems that generalize the tools previously given to
establish $H$-consistency bounds. This generalization proves essential for
analyzing $H$-consistency bounds specific to regression. Next, we prove a
series of novel $H$-consistency bounds for surrogate loss functions of the
squared loss, under the assumption of a symmetric distribution and a bounded
hypothesis set. This includes positive results for the Huber loss, all $\ell_p$
losses, $p \geq 1$, the squared $\epsilon$-insensitive loss, as well as a
negative result for the $\epsilon$-insensitive loss used in squared Support
Vector Regression (SVR). We further leverage our analysis of $H$-consistency
for regression and derive principled surrogate losses for adversarial
regression (Section 5). This readily establishes novel algorithms for
adversarial regression, for which we report favorable experimental results in
Section 6.

我们对回归的 H 一致性界限进行了详细研究，提出了适用于回归的 H 一致性界限的新定理，并针对对称分布和有界假设集的平方损失函数的代理损失函数证明了一系列新的 H 一致性界限，包括 Huber 损失、所有的 lp 损失 (p≥1)、平方 ε-insensitive 损失，以及在平方支持向量回归 (SVR) 中使用的 ε-insensitive 损失的负面结果。我们进一步利用我们对回归 H 一致性的分析，并推导出对抗性回归的合理代理损失函数，从而建立了新的对抗性回归算法，并在第 6 节报道了有利的实验结果。

$H$- 一致性保证用于回归

$H$-Consistency Guarantees for Regression

We present a detailed study of top-$k$ classification, the task of predicting
the $k$ most probable classes for an input, extending beyond single-class
prediction. We demonstrate that several prevalent surrogate loss functions in
multi-class classification, such as comp-sum and constrained losses, are
supported by $H$-consistency bounds with respect to the top-$k$ loss. These
bounds guarantee consistency in relation to the hypothesis set $H$, providing
stronger guarantees than Bayes-consistency due to their non-asymptotic and
hypothesis-set specific nature. To address the trade-off between accuracy and
cardinality $k$, we further introduce cardinality-aware loss functions through
instance-dependent cost-sensitive learning. For these functions, we derive
cost-sensitive comp-sum and constrained surrogate losses, establishing their
$H$-consistency bounds and Bayes-consistency. Minimizing these losses leads to
new cardinality-aware algorithms for top-$k$ classification. We report the
results of extensive experiments on CIFAR-100, ImageNet, CIFAR-10, and SVHN
datasets demonstrating the effectiveness and benefit of these algorithms.

我们研究了 top-k 分类的细节，该任务是预测输入的 $k$ 个可能的类别，超越单一分类预测。我们证明了多类别分类中的几种普遍代理损失函数，如 comp-sum 和约束损失，具有相对于 top-k 损失的 H 一致性界限。这些界限保证了与假设集 $H$ 相关的一致性，提供比贝叶斯一致性更强的保证，因为它们是非渐近和假设集特定的。为了解决准确性和基数 $k$ 之间的权衡，我们进一步通过实例依赖的成本敏感学习引入了基数感知的损失函数。对于这些函数，我们得到了成本敏感的 comp-sum 和约束代理损失，建立了它们的 H 一致性界限和贝叶斯一致性。最小化这些损失会导致新的基数感知算法应用于 top-k 分类。我们报道了在 CIFAR-100、ImageNet、CIFAR-10 和 SVHN 数据集上的大量实验证明了这些算法的有效性和好处。

Top-k 分类与基数感知预测

Top-$k$ Classification and Cardinality-Aware Prediction

Cross-entropy is a widely used loss function in applications. It coincides
with the logistic loss applied to the outputs of a neural network, when the
softmax is used. But, what guarantees can we rely on when using cross-entropy
as a surrogate loss? We present a theoretical analysis of a broad family of
losses, comp-sum losses, that includes cross-entropy (or logistic loss),
generalized cross-entropy, the mean absolute error and other loss
cross-entropy-like functions. We give the first $H$-consistency bounds for
these loss functions. These are non-asymptotic guarantees that upper bound the
zero-one loss estimation error in terms of the estimation error of a surrogate
loss, for the specific hypothesis set $H$ used. We further show that our bounds
are tight. These bounds depend on quantities called minimizability gaps, which
only depend on the loss function and the hypothesis set. To make them more
explicit, we give a specific analysis of these gaps for comp-sum losses. We
also introduce a new family of loss functions, smooth adversarial comp-sum
losses, derived from their comp-sum counterparts by adding in a related smooth
term. We show that these loss functions are beneficial in the adversarial
setting by proving that they admit $H$-consistency bounds. This leads to new
adversarial robustness algorithms that consist of minimizing a regularized
smooth adversarial comp-sum loss. While our main purpose is a theoretical
analysis, we also present an extensive empirical analysis comparing comp-sum
losses. We further report the results of a series of experiments demonstrating
that our adversarial robustness algorithms outperform the current
state-of-the-art, while also achieving a superior non-adversarial accuracy.

本文研究了广泛应用的交叉熵损失函数，提出了一族损失函数 comp-sum，包括了交叉熵、广义交叉熵、平均绝对误差等。我们首次给出了这些损失函数的 H - 相容性，进一步介绍了一种新的平滑对抗 comp-sum 损失函数，并证明了它们有助于在对抗性环境下提高模型的 H - 相容性。