Vector embeddings have become ubiquitous tools for many language-related
tasks. A leading embedding model is OpenAI's text-ada-002 which can embed
approximately 6,000 words into a 1,536-dimensional vector. While powerful,
text-ada-002 is not open source and is only available via API. We trained a
simple neural network to convert open-source 768-dimensional MPNet embeddings
into text-ada-002 embeddings. We compiled a subset of 50,000 online food
reviews. We calculated MPNet and text-ada-002 embeddings for each review and
trained a simple neural network to for 75 epochs. The neural network was
designed to predict the corresponding text-ada-002 embedding for a given MPNET
embedding. Our model achieved an average cosine similarity of 0.932 on 10,000
unseen reviews in our held-out test dataset. We manually assessed the quality
of our predicted embeddings for vector search over text-ada-002-embedded
reviews. While not as good as real text-ada-002 embeddings, predicted
embeddings were able to retrieve highly relevant reviews. Our final model,
Vec2Vec, is lightweight (<80 MB) and fast. Future steps include training a
neural network with a more sophisticated architecture and a larger dataset of
paired embeddings to achieve greater performance. The ability to convert
between and align embedding spaces may be helpful for interoperability,
limiting dependence on proprietary models, protecting data privacy, reducing
costs, and offline operations.

本研究提出了一个名为 Vec2Vec 的网络模型，它可以将开源的 768 维 MPNet 嵌入转换为 1,536 维的 text-ada-002 嵌入，其平均余弦相似度可达 0.932，并且可以用于以更低成本、更高效的方式进行搜索和数据隐私保护。