In this paper, we present a new embedding model, called M3-Embedding, which
is distinguished for its versatility in Multi-Linguality, Multi-Functionality,
and Multi-Granularity. It can support more than 100 working languages, leading
to new state-of-the-art performances on multi-lingual and cross-lingual
retrieval tasks. It can simultaneously perform the three common retrieval
functionalities of embedding model: dense retrieval, multi-vector retrieval,
and sparse retrieval, which provides a unified model foundation for real-world
IR applications. It is able to process inputs of different granularities,
spanning from short sentences to long documents of up to 8192 tokens. The
effective training of M3-Embedding involves the following technical
contributions. We propose a novel self-knowledge distillation approach, where
the relevance scores from different retrieval functionalities can be integrated
as the teacher signal to enhance the training quality. We also optimize the
batching strategy, enabling a large batch size and high training throughput to
ensure the discriminativeness of embeddings. To the best of our knowledge,
M3-Embedding is the first embedding model which realizes such a strong
versatility. The model and code will be publicly available at
this https URL

本文介绍了一种新的嵌入模型 M3-Embedding，其以其多语、多功能和多粒度的通用性而脱颖而出。它支持 100 多种工作语言，展示了在多语言和跨语言检索任务上的最新性能，同时能够同时执行三种常见检索功能：密集检索、多向量检索和稀疏检索，为现实世界的信息检索应用提供了统一的模型基础。该模型能够处理不同粒度的输入，从短句到最长 8192 个标记的长文档。我们提出了一种新颖的自知识蒸馏方法，通过集成来自不同检索功能的相关性得分作为教师信号来增强训练质量，并优化了批处理策略，以确保嵌入的区分性。据我们所知，M3-Embedding 是第一个实现如此强大通用性的嵌入模型。该模型和代码将公开在指定的 URL 链接中。