Pre-trained vision and language models such as CLIP have witnessed remarkable
success in connecting images and texts with a primary focus on English texts.
Despite recent efforts to extend CLIP to support other languages, disparities
in performance among different languages have been observed due to uneven
resource availability. Additionally, current cross-lingual transfer methods of
those pre-trained models would consume excessive resources for a large number
of languages. Therefore, we propose a new parameter-efficient cross-lingual
transfer learning framework that utilizes a translation-based alignment method
to mitigate multilingual disparities and explores parameter-efficient
fine-tuning methods for parameter-efficient cross-lingual transfer. Extensive
experiments on XTD and Multi30K datasets, covering 11 languages under
zero-shot, few-shot, and full-dataset learning scenarios, show that our
framework significantly reduces the multilingual disparities among languages
and improves cross-lingual transfer results, especially in low-resource
scenarios, while only keeping and fine-tuning an extremely small number of
parameters compared to the full model (e.g., Our framework only requires 0.16\%
additional parameters of a full-model for each language in the few-shot
learning scenario).

本文提出一种新的参数高效的跨语言转移学习框架，利用基于翻译的对齐方法来缓解多语言差异，并探索参数高效的微调方法，经过广泛的实验表明，我们的框架显著减少了语言之间的多语言差异，并在跨语言转移方面取得了改进，尤其在低资源场景中，同时只保留和微调极少量的参数与全模型相比（例如，我们的框架对于每种语言只需要全模型的 0.16% 的额外参数，在 few-shot 学习场景下）。