Pre-trained language models have shown excellent results in few-shot learning
scenarios using in-context learning. Although it is impressive, the size of
language models can be prohibitive to make them usable in on-device
applications, such as sensors or smartphones. With smaller language models,
task-specific data annotation is needed to fine-tune the language model for a
specific purpose. However, data annotation can have a substantial financial and
time burden for small research groups, startups, and even companies. In this
paper, we analyze different prompt-based fine-tuning techniques to improve
results on both language and multimodal causal transformer models. To evaluate
our results, we use a dataset focusing on visual commonsense reasoning in time.
Our results show that by simple model-agnostic prompt-based fine-tuning,
comparable results can be reached by only using 35%-40% of the fine-tuning
training dataset. The proposed approaches result in significant time and
financial savings. As the proposed methods make minimal architectural
assumptions, other researchers can use the results in their transformer models
with minimal adaptations. We plan to release the source code freely to make it
easier for the community to use and contribute to our work.

本文研究如何使用 Prompt-based Fine-tuning 技术提高语言模型和多模式因果变换器模型的效果，结果表明使用只有 35%-40% 的训练数据集便能取得可比较的效果，从而达到显著的时间和费用节约。