Feb, 2024
视觉语言模型中的 Déjà Vu 记忆
Déjà Vu Memorization in Vision-Language Models
Bargav Jayaraman, Chuan Guo, Kamalika Chaudhuri
TL;DRVision-Language Models have been widely used with downstream applications, and this paper proposes a method for measuring memorization in these models called déjà vu memorization, confirming its significance in OpenCLIP trained on image-caption pairs and showing that text randomization mitigates the extent of memorization while moderately affecting downstream task performance.