Embedding models are integral to AI applications like semantic search,
personalized recommendations, and retrieval augmented generation for LLMs,
necessitating high-quality training data. However, the limited scalability of
manual data curation prompts the need for automated methods to ensure data
integrity. Traditional unsupervised triplet mining automates training data
generation, crucial for embedding model training, yet inadvertently injects
biases and noise, thereby degrading model performance. Addressing this, we
introduce GISTEmbed, a novel strategy that enhances in-batch negative selection
during contrastive training through a guide model. This approach departs from
reliance on random sampling and equal utility assumption of batch negatives,
significantly reducing noise from data quality issues and improving model
fine-tuning. Benchmarked against the Massive Text Embedding Benchmark (MTEB),
GISTEmbed showcases consistent performance improvements across various model
sizes and achieves state-of-the-art results in select categories. This
framework enables significant enhancements for smaller models by leveraging the
capabilities of powerful yet resource-intensive large models. GISTEmbed can
potentially revolutionize the creation of highly efficient, smaller models,
democratizing access to advanced AI technologies. Making these technologies
more accessible and cost-effective, especially for applications constrained by
resources, significantly expands the impact and accessibility of
state-of-the-art AI solutions across diverse sectors.

通过引入 GISTEmbed，在对比训练中通过导向模型增强批内负例选择，从而显著减少数据质量问题引入的噪声，并改善模型微调，从而提供了对规模较小模型的显著改进的框架，借助资源密集型的大型模型的能力，GISTEmbed 有可能革新高效且规模较小模型的创建方法，使得先进的 AI 技术更加可用和具有可承担性，从而在不同领域大大扩展了最先进的 AI 解决方案的影响和可及性。

GISTEmbed: 文本嵌入微调中的引导样本选择

GISTEmbed: Guided In-sample Selection of Training Negatives for Text  Embedding Fine-tuning

We analyze the effect of offline and online triplet mining for colorectal
cancer (CRC) histopathology dataset containing 100,000 patches. We consider the
extreme, i.e., farthest and nearest patches to a given anchor, both in online
and offline mining. While many works focus solely on selecting the triplets
online (batch-wise), we also study the effect of extreme distances and neighbor
patches before training in an offline fashion. We analyze extreme cases'
impacts in terms of embedding distance for offline versus online mining,
including easy positive, batch semi-hard, batch hard triplet mining,
neighborhood component analysis loss, its proxy version, and distance weighted
sampling. We also investigate online approaches based on extreme distance and
comprehensively compare offline, and online mining performance based on the
data patterns and explain offline mining as a tractable generalization of the
online mining with large mini-batch size. As well, we discuss the relations of
different colorectal tissue types in terms of extreme distances. We found that
offline and online mining approaches have comparable performances for a
specific architecture, such as ResNet-18 in this study. Moreover, we found the
assorted case, including different extreme distances, is promising, especially
in the online approach.

研究了离线和在线三元组挖掘在包含 100,000 个样本的结肠直肠癌组织学数据集中的效果，并分析了它们在嵌入距离和性能方面的影响，离线和在线挖掘在特定体系结构下具有可比性。

基于极端距离的组织病理图像离线和在线三元组挖掘比较

Offline versus Online Triplet Mining based on Extreme Distances of  Histopathology Patches

Despite significant recent advances in the field of face recognition,
implementing face verification and recognition efficiently at scale presents
serious challenges to current approaches. In this paper we present a system,
called FaceNet, that directly learns a mapping from face images to a compact
Euclidean space where distances directly correspond to a measure of face
similarity. Once this space has been produced, tasks such as face recognition,
verification and clustering can be easily implemented using standard techniques
with FaceNet embeddings as feature vectors.
Our method uses a deep convolutional network trained to directly optimize the
embedding itself, rather than an intermediate bottleneck layer as in previous
deep learning approaches. To train, we use triplets of roughly aligned matching
/ non-matching face patches generated using a novel online triplet mining
method. The benefit of our approach is much greater representational
efficiency: we achieve state-of-the-art face recognition performance using only
128-bytes per face.
On the widely used Labeled Faces in the Wild (LFW) dataset, our system
achieves a new record accuracy of 99.63%. On YouTube Faces DB it achieves
95.12%. Our system cuts the error rate in comparison to the best published
result by 30% on both datasets.
We also introduce the concept of harmonic embeddings, and a harmonic triplet
loss, which describe different versions of face embeddings (produced by
different networks) that are compatible to each other and allow for direct
comparison between each other.

本文介绍了一种名为 FaceNet 的系统，其通过使用卷积神经网络及三元组学习法直接将人脸图像映射到紧凑的欧几里得空间中，使得人脸识别、验证和聚类等任务可使用 FaceNet 嵌入作为特征向量，仅使用 128 字节每个人脸即可实现最先进的人脸识别性能，其在 LFW 和 YouTube Faces DB 两个数据集上的表现都创造了最新纪录。同时，本文还提出了谐波嵌入的概念和谐波三元组损失，以描述不同网络产生的面部嵌入版本以及它们之间的直接比较。