Large Language Models (LLMs) have transformed NLP with their remarkable
In-context Learning (ICL) capabilities. Automated assistants based on LLMs are
gaining popularity; however, adapting them to novel tasks is still challenging.
While colossal models excel in zero-shot performance, their computational
demands limit widespread use, and smaller language models struggle without
context. This paper investigates whether LLMs can generalize from labeled
examples of predefined tasks to novel tasks. Drawing inspiration from
biological neurons and the mechanistic interpretation of the Transformer
architecture, we explore the potential for information sharing across tasks. We
design a cross-task prompting setup with three LLMs and show that LLMs achieve
significant performance improvements despite no examples from the target task
in the context. Cross-task prompting leads to a remarkable performance boost of
107% for LLaMA-2 7B, 18.6% for LLaMA-2 13B, and 3.2% for GPT 3.5 on average
over zero-shot prompting, and performs comparable to standard in-context
learning. The effectiveness of generating pseudo-labels for in-task examples is
demonstrated, and our analyses reveal a strong correlation between the effect
of cross-task examples and model activation similarities in source and target
input tokens. This paper offers a first-of-its-kind exploration of LLMs'
ability to solve novel tasks based on contextual signals from different task
examples.

该研究论文探讨了大型语言模型 (LLMs) 是否能够根据不同任务示例的上下文信号来解决新任务，并设计了一个跨任务提示设置，并表明 LLMs 在无需上下文提示的情况下能够获得显著的性能提升，同时展示了模型激活相似性与跨任务示例效果之间的强相关性。