BriefGPT.xyz
May, 2024
NV-Embed: LLM 训练通用嵌入模型的改进技术
NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Models
HTML
PDF
Chankyu Lee, Rajarshi Roy, Mengyao Xu, Jonathan Raiman, Mohammad Shoeybi...
TL;DR
通过引入各种架构设计和训练过程,NV-Embed模型显著提高了LLM作为多功能嵌入模型的性能,同时保持其简单性和可重现性,并取得了69.32的记录高分,在包括检索、重排序、分类、聚类和语义文本相似性任务在内的56个任务中名列第一。
Abstract
decoder-only large language model
(LLM)-based embedding models are beginning to outperform BERT or T5-based embedding models in general-purpose text embedding tasks, including dense vector-based retrieval. In this work, we introduce the
→