Recent advances in multimodal learning has resulted in powerful
vision-language models, whose representations are generalizable across a
variety of downstream tasks. Recently, their generalizability has been further
extended by incorporating trainable prompts, borrowed from the natural language
processing literature. While such prompt learning techniques have shown
impressive results, we identify that these prompts are trained based on global
image features which limits itself in two aspects: First, by using global
features, these prompts could be focusing less on the discriminative foreground
image, resulting in poor generalization to various out-of-distribution test
cases. Second, existing work weights all prompts equally whereas our intuition
is that these prompts are more specific to the type of the image. We address
these issues with as part of our proposed Contextual Prompt Learning (CoPL)
framework, capable of aligning the prompts to the localized features of the
image. Our key innovations over earlier works include using local image
features as part of the prompt learning process, and more crucially, learning
to weight these prompts based on local features that are appropriate for the
task at hand. This gives us dynamic prompts that are both aligned to local
image features as well as aware of local contextual relationships. Our
extensive set of experiments on a variety of standard and few-shot datasets
show that our method produces substantially improved performance when compared
to the current state of the art methods. We also demonstrate both few-shot and
out-of-distribution performance to establish the utility of learning dynamic
prompts that are aligned to local image features.

本文提出一种叫做上下文提示学习的框架，用于多模态学习中训练可对齐图像本地化特征并且能够适应当前任务的动态提示，通过应用于多个标准和少样本数据集上展示了比现有技术方法更优越的性能和强大的适用性。

视觉语言理解的情境提示学习

Contextual Prompt Learning for Vision-Language Understanding

Despite its astounding success in learning deeper multi-dimensional data, the
performance of deep learning declines on new unseen tasks mainly due to its
focus on same-distribution prediction. Moreover, deep learning is notorious for
poor generalization from few samples. Meta-learning is a promising approach
that addresses these issues by adapting to new tasks with few-shot datasets.
This survey first briefly introduces meta-learning and then investigates
state-of-the-art meta-learning methods and recent advances in: (I)
metric-based, (II) memory-based, (III), and learning-based methods. Finally,
current challenges and insights for future researches are discussed.

该论文介绍了元学习在深度学习中的应用及其近期发展，涵盖度量、记忆、基于学习等方法，并探讨了当前的挑战和未来的研究方向。