In this work, we introduce a new unsupervised embedding method, Meta-Task
Prompting with Explicit One-Word Limitation (MetaEOL), for generating
high-quality sentence embeddings from Large Language Models (LLMs) without the
need for model fine-tuning or task-specific engineering. Leveraging meta-task
prompting, MetaEOL guides LLMs to produce embeddings through a series of
carefully designed prompts that address multiple representational aspects. Our
comprehensive experiments demonstrate that embeddings averaged from various
meta-tasks yield competitive performance on Semantic Textual Similarity (STS)
benchmarks and excel in downstream tasks, surpassing contrastive-trained
models. Our findings suggest a new scaling law for embedding generation,
offering a versatile, resource-efficient approach for embedding extraction
across diverse sentence-centric scenarios.

通过使用一种新的无监督嵌入方法，MetaEOL，我们能够从大型语言模型（LLMs）中生成高质量的句子嵌入，而不需要模型微调或任务特定的工程处理。使用元任务提示，MetaEOL 通过一系列精心设计的提示引导 LLMs 生成嵌入，这些提示处理了多个表示方面。我们的综合实验表明，从各种元任务平均得到的嵌入在语义文本相似性（STS）基准测试中表现出了竞争力，并在下游任务中表现优异，并超越了对比训练的模型。我们的发现提出了一种新的嵌入生成规律，并提供了一种多用途、节约资源的嵌入提取方法，适用于不同的句子中心场景。