In this work, we introduce a new unsupervised embedding method, Meta-Task
Prompting with Explicit One-Word Limitation (MetaEOL), for generating
high-quality sentence embeddings from Large Language Models (LLMs) without the
need for model fine-tuning or task-specific engineering. Leveraging meta-task
prompting, MetaEOL guides LLMs to produce embeddings through a series of
carefully designed prompts that address multiple representational aspects. Our
comprehensive experiments demonstrate that embeddings averaged from various
meta-tasks yield competitive performance on Semantic Textual Similarity (STS)
benchmarks and excel in downstream tasks, surpassing contrastive-trained
models. Our findings suggest a new scaling law for embedding generation,
offering a versatile, resource-efficient approach for embedding extraction
across diverse sentence-centric scenarios.

通过使用一种新的无监督嵌入方法，MetaEOL，我们能够从大型语言模型（LLMs）中生成高质量的句子嵌入，而不需要模型微调或任务特定的工程处理。使用元任务提示，MetaEOL 通过一系列精心设计的提示引导 LLMs 生成嵌入，这些提示处理了多个表示方面。我们的综合实验表明，从各种元任务平均得到的嵌入在语义文本相似性（STS）基准测试中表现出了竞争力，并在下游任务中表现优异，并超越了对比训练的模型。我们的发现提出了一种新的嵌入生成规律，并提供了一种多用途、节约资源的嵌入提取方法，适用于不同的句子中心场景。

元任务提示引发大型语言模型的嵌入

Meta-Task Prompting Elicits Embedding from Large Language Models

Summarization for scientific text has shown significant benefits both for the
research community and human society. Given the fact that the nature of
scientific text is distinctive and the input of the multi-document
summarization task is substantially long, the task requires sufficient
embedding generation and text truncation without losing important information.
To tackle these issues, in this paper, we propose SKT5SciSumm - a hybrid
framework for multi-document scientific summarization (MDSS). We leverage the
Sentence-Transformer version of Scientific Paper Embeddings using
Citation-Informed Transformers (SPECTER) to encode and represent textual
sentences, allowing for efficient extractive summarization using k-means
clustering. We employ the T5 family of models to generate abstractive summaries
using extracted sentences. SKT5SciSumm achieves state-of-the-art performance on
the Multi-XScience dataset. Through extensive experiments and evaluation, we
showcase the benefits of our model by using less complicated models to achieve
remarkable results, thereby highlighting its potential in advancing the field
of multi-document summarization for scientific text.

SKT5SciSumm 是一个混合框架，利用 SPECTER 对文本句子进行编码和表示，使用 k-means 聚类进行提取式摘要，利用 T5 系列模型生成抽象总结，从而在科学文本的多文档摘要任务上取得了最新的成果。