The ability to use the same distance threshold across different test classes
/ distributions is highly desired for a frictionless deployment of commercial
image retrieval systems. However, state-of-the-art deep metric learning losses
often result in highly varied intra-class and inter-class embedding structures,
making threshold calibration a non-trivial process in practice. In this paper,
we propose a novel metric named Operating-Point-Incosistency-Score (OPIS) that
measures the variance in the operating characteristics across different classes
in a target calibration range, and demonstrate that high accuracy of a metric
learning embedding model does not guarantee calibration consistency for both
seen and unseen classes. We find that, in the high-accuracy regime, there
exists a Pareto frontier where accuracy improvement comes at the cost of
calibration consistency. To address this, we develop a novel regularization,
named Calibration-Aware Margin (CAM) loss, to encourage uniformity in the
representation structures across classes during training. Extensive experiments
demonstrate CAM's effectiveness in improving calibration-consistency while
retaining or even enhancing accuracy, outperforming state-of-the-art deep
metric learning methods.

使用相同的距离阈值将不同的测试类别 / 分布应用于商业图像检索系统的能力是非常可取的；然而，最先进的深度度量学习损失常常导致高度变化的类内和类间嵌入结构，使得阈值校准在实践中成为一个非常复杂的过程。本文提出了一种名为操作点不一致性分数（OPIS）的新度量，用于衡量目标校准范围内不同类别之间操作特性的方差，并且证明度量学习嵌入模型的高准确性不能保证对已见和未见类别的校准一致性。我们发现，在高准确性区域存在一条帕累托前沿，在提高准确性的同时牺牲校准一致性。为了解决这个问题，我们开发了一种名为校准感知边界（CAM）损失的新型正则化方法，以在训练过程中鼓励类别间表示结构的一致性。大量实验证明了 CAM 在提升校准一致性的同时保持甚至提高准确性方面的有效性，并且优于最先进的深度度量学习方法。