Deploying deep visual models can lead to performance drops due to the
discrepancies between source and target distributions. Several approaches
leverage labeled source data to estimate target domain accuracy, but accessing
labeled source data is often prohibitively difficult due to data
confidentiality or resource limitations on serving devices. Our work proposes a
new framework to estimate model accuracy on unlabeled target data without
access to source data. We investigate the feasibility of using pseudo-labels
for accuracy estimation and evolve this idea into adopting recent advances in
source-free domain adaptation algorithms. Our approach measures the
disagreement rate between the source hypothesis and the target pseudo-labeling
function, adapted from the source hypothesis. We mitigate the impact of
erroneous pseudo-labels that may arise due to a high ideal joint hypothesis
risk by employing adaptive adversarial perturbation on the input of the target
model. Our proposed source-free framework effectively addresses the challenging
distribution shift scenarios and outperforms existing methods requiring source
data and labels for training.

无需访问源数据，本研究提出了一种新的框架，通过估计无标签目标数据的模型准确性来解决深度视觉模型在源和目标分布之间存在性能降低的问题。将伪标签用于准确性估计，并采用最新的无源域自适应算法，利用源假设来适应目标伪标签函数，并通过对目标模型的输入进行自适应对抗扰动以减轻错误伪标签的影响，从而有效解决具有挑战性的分布偏移情景，并胜过需要源数据和标签进行训练的现有方法。

无需源样本的领域自适应对抗扰动法深度视觉模型无监督准确度估计

Unsupervised Accuracy Estimation of Deep Visual Models using  Domain-Adaptive Adversarial Perturbation without Source Samples

Ensembling has a long history in statistical data analysis, with many
impactful applications. However, in many modern machine learning settings, the
benefits of ensembling are less ubiquitous and less obvious. We study, both
theoretically and empirically, the fundamental question of when ensembling
yields significant performance improvements in classification tasks.
Theoretically, we prove new results relating the \emph{ensemble improvement
rate} (a measure of how much ensembling decreases the error rate versus a
single model, on a relative scale) to the \emph{disagreement-error ratio}. We
show that ensembling improves performance significantly whenever the
disagreement rate is large relative to the average error rate; and that,
conversely, one classifier is often enough whenever the disagreement rate is
low relative to the average error rate. On the way to proving these results, we
derive, under a mild condition called \emph{competence}, improved upper and
lower bounds on the average test error rate of the majority vote classifier. To
complement this theory, we study ensembling empirically in a variety of
settings, verifying the predictions made by our theory, and identifying
practical scenarios where ensembling does and does not result in large
performance improvements. Perhaps most notably, we demonstrate a distinct
difference in behavior between interpolating models (popular in current
practice) and non-interpolating models (such as tree-based methods, where
ensembling is popular), demonstrating that ensembling helps considerably more
in the latter case than in the former.

研究加入集成（ensembling）技术在分类任务中的性能提升问题，证明当集成模型中错误率低于集成模型间的不同率时，在学习任务中加入集成技术可以显著提高性能。通过理论和实验的方式证明这种推论，并在实践中找出集成技术适用和不适用的情况。