In deep metric learning, the Triplet Loss has emerged as a popular method to
learn many computer vision and natural language processing tasks such as facial
recognition, object detection, and visual-semantic embeddings. One issue that
plagues the Triplet Loss is network collapse, an undesirable phenomenon where
the network projects the embeddings of all data onto a single point.
Researchers predominately solve this problem by using triplet mining
strategies. While hard negative mining is the most effective of these
strategies, existing formulations lack strong theoretical justification for
their empirical success. In this paper, we utilize the mathematical theory of
isometric approximation to show an equivalence between the Triplet Loss sampled
by hard negative mining and an optimization problem that minimizes a
Hausdorff-like distance between the neural network and its ideal counterpart
function. This provides the theoretical justifications for hard negative
mining's empirical efficacy. In addition, our novel application of the
isometric approximation theorem provides the groundwork for future forms of
hard negative mining that avoid network collapse. Our theory can also be
extended to analyze other Euclidean space-based metric learning methods like
Ladder Loss or Contrastive Learning.

本文讨论了利用等距逼近理论解决深度度量学习中 Triplet Loss 网络崩塌问题以及 Hard Negative Mining 的理论依据，并提出了未来避免网络崩塌的 Hard Negative Mining 方法。此理论还可拓展至其他基于欧几里得空间的度量学习方法。