We investigate whether prompts learned independently for different tasks can
be later combined through prompt algebra to obtain a model that supports
composition of tasks. We consider Visual Language Models (VLM) with prompt
tuning as our base classifier and formally define the notion of prompt algebra.
We propose constrained prompt tuning to improve performance of the composite
classifier. In the proposed scheme, prompts are constrained to appear in the
lower dimensional subspace spanned by the basis vectors of the pre-trained
vocabulary. Further regularization is added to ensure that the learned prompt
is grounded correctly to the existing pre-trained vocabulary. We demonstrate
the effectiveness of our method on object classification and object-attribute
classification datasets. On average, our composite model obtains classification
accuracy within 2.5% of the best base model. On UTZappos it improves
classification accuracy over the best base model by 8.45% on average.

本文探讨使用独立学习的提示符是否可以通过提示代数，组合不同任务的模型。我们考虑使用提示调整的视觉语言模型 (VLM) 作为我们的基本分类器，形式化地定义提示代数的概念，提出了约束提示调整来改善复合分类器的性能。我们证明了我们方法的有效性对象分类和属性分类数据集，我们的复合模型平均准确度与最佳基本模型相差不到 2.5％，在 UTZappos 上，平均提高了 8.45％分类准确率。