Large pre-trained models have proved to be remarkable zero- and
(prompt-based) few-shot learners in unimodal vision and language tasks. We
propose MAPL, a simple and parameter-efficient method that reuses frozen
pre-trained unimodal models and leverages their strong generalization
capabilities in multimodal vision-language (VL) settings. MAPL learns a
lightweight mapping between the representation spaces of unimodal models using
aligned image-text data, and can generalize to unseen VL tasks from just a few
in-context examples. The small number of trainable parameters makes MAPL
effective at low-data and in-domain learning. Moreover, MAPL's modularity
enables easy extension to other pre-trained models. Extensive experiments on
several visual question answering and image captioning benchmarks show that
MAPL achieves superior or competitive performance compared to similar methods
while training orders of magnitude fewer parameters. MAPL can be trained in
just a few hours using modest computational resources and public datasets. We
release our code and pre-trained model weights at
this https URL

MAPL 是一种有效的参数少、可复用预训练模型并利用其在多模态视觉语言领域中的强大泛化能力的方法，能够将不同模态的模型的表示空间通过对齐的图像文本数据进行轻量级映射，从而在现场学习过程中减少训练量并产生较好的性能。