The conventional evaluation protocols on machine learning models rely heavily on a labeled, i.i.d-assumed testing dataset, which is not often present in real world applications. The Automated Model Evaluation (AutoEval) shows an alternative to this traditional workflow, by forming a proximal prediction pipeline of the testing performance without the presence of ground-truth labels. Despite its recent successes, the AutoEval frameworks still suffer from an overconfidence issue, substantial storage and computational cost. In that regard, we propose a novel measure -- Meta-Distribution Energy (MDE) -- that allows the AutoEval framework to be both more efficient and effective. The core of the MDE is to establish a meta-distribution statistic, on the information (energy) associated with individual samples, then offer a smoother representation enabled by energy-based learning. We further provide our theoretical insights by connecting the MDE with the classification loss. We provide extensive experiments across modalities, datasets and different architectural backbones to validate MDE's validity, together with its superiority compared with prior approaches. We also prove MDE's versatility by showing its seamless integration with large-scale models, and easy adaption to learning scenarios with noisy- or imbalanced- labels.

提出一种新的度量 - 元分布能量 (MDE)，用于提高自动模型评估框架的效率和效果，通过建立与个体样本相关的信息（能量）的元分布统计，并借助基于能量的学习提供更平滑的表示。验证了MDE在多模态、不同数据集和不同架构背景下的有效性以及与之前方法相比的优越性，并证明了MDE与分类损失的理论联系。同时展示了MDE与大规模模型的无缝集成以及在具有噪声或不平衡标签的学习场景中的简便适应性。

基于能量的自动模型评估