With the booming of Large Language Models (LLMs), prompt-learning has become
a promising method mainly researched in various research areas. Recently, many
attempts based on prompt-learning have been made to improve the performance of
text classification. However, most of these methods are based on heuristic
Chain-of-Thought (CoT), and tend to be more complex but less efficient. In this
paper, we rethink the LLM-based text classification methodology, propose a
simple and effective transfer learning strategy, namely LLMEmbed, to address
this classical but challenging task. To illustrate, we first study how to
properly extract and fuse the text embeddings via various lightweight LLMs at
different network depths to improve their robustness and discrimination, then
adapt such embeddings to train the classifier. We perform extensive experiments
on publicly available datasets, and the results show that LLMEmbed achieves
strong performance while enjoys low training overhead using lightweight LLM
backbones compared to recent methods based on larger LLMs, i.e. GPT-3, and
sophisticated prompt-based strategies. Our LLMEmbed achieves adequate accuracy
on publicly available benchmarks without any fine-tuning while merely use 4%
model parameters, 1.8% electricity consumption and 1.5% runtime compared to its
counterparts. Code is available at:
this https URL

通过轻量级的大型语言模型，我们提出一种简单而有效的迁移学习策略 LLMEmbed 来改善文本分类的性能，并在公开数据集上的广泛实验验证了其强大的性能，相较于基于更大型的语言模型（如 GPT-3）和复杂的提示策略，我们的方法仅使用 4% 的模型参数、1.8% 的能耗和 1.5% 的运行时间即可达到充分的准确性。