Retrieval Augmented Generation (RAG) has emerged as an effective solution for
mitigating hallucinations in Large Language Models (LLMs). The retrieval stage
in RAG typically involves a pre-trained embedding model, which converts queries
and passages into vectors to capture their semantics. However, a standard
pre-trained embedding model may exhibit sub-optimal performance when applied to
specific domain knowledge, necessitating fine-tuning. This paper addresses
scenarios where the embeddings are only available from a black-box model. We
introduce Model augmented fine-tuning (Mafin) -- a novel approach for
fine-tuning a black-box embedding model by augmenting it with a trainable
embedding model. Our results demonstrate that Mafin significantly enhances the
performance of the black-box embeddings by only requiring the training of a
small augmented model. We validate the effectiveness of our method on both
labeled and unlabeled datasets, illustrating its broad applicability and
efficiency.

我们介绍了一种新的方法，利用可训练的嵌入模型对黑盒嵌入模型进行微调，以显著提高其性能，并验证了该方法在有标签和无标签数据集上的有效性，展示了其广泛适用性和高效性。