Large pretrained language models (LLMs) can be rapidly adapted to a wide variety of tasks via a text-to-text approach, where the instruction and input are fed to the model in natural language. Combined with in-context learning (ICL), this paradigm is impressively flexible and powerful. However, it also burdens users with an overwhelming number of choices, many of them arbitrary. Inspired by markup languages like HTML, we contribute a method of using soft-token tags to compose prompt templates. This approach reduces arbitrary decisions and streamlines the application of ICL. Our method is a form of meta-learning for ICL; it learns these tags in advance during a parameter-efficient fine-tuning ``warm-up'' process. The tags can subsequently be used in templates for ICL on new, unseen tasks without any additional fine-tuning. Our experiments with this approach yield promising initial results, improving LLM performance on important enterprise applications such as few-shot and open-world intent detection, as well as text classification in news and legal domains.

通过使用软标记标签来组成提示模板的方法，以及预先学习这些标签的元学习方法，可以减少用户在ICL中的任意决策并简化ICL的应用。该方法在企业应用中的初步实验结果显示出了潜在的改进，对于少样本和开放世界意图检测以及新闻和法律领域的文本分类有良好的表现。

ICL标记：使用软词标签进行上下文学习的结构化