Machine unlearning empowers individuals with the `right to be forgotten' by
removing their private or sensitive information encoded in machine learning
models. However, it remains uncertain whether MU can be effectively applied to
Multimodal Large Language Models (MLLMs), particularly in scenarios of
forgetting the leaked visual data of concepts. To overcome the challenge, we
propose an efficient method, Single Image Unlearning (SIU), to unlearn the
visual recognition of a concept by fine-tuning a single associated image for
few steps. SIU consists of two key aspects: (i) Constructing Multifaceted
fine-tuning data. We introduce four targets, based on which we construct
fine-tuning data for the concepts to be forgotten; (ii) Jointly training loss.
To synchronously forget the visual recognition of concepts and preserve the
utility of MLLMs, we fine-tune MLLMs through a novel Dual Masked KL-divergence
Loss combined with Cross Entropy loss. Alongside our method, we establish
MMUBench, a new benchmark for MU in MLLMs and introduce a collection of metrics
for its evaluation. Experimental results on MMUBench show that SIU completely
surpasses the performance of existing methods. Furthermore, we surprisingly
find that SIU can avoid invasive membership inference attacks and jailbreak
attacks. To the best of our knowledge, we are the first to explore MU in MLLMs.
We will release the code and benchmark in the near future.

通过建立大规模多模态语言模型（Multimodal Large Language Models）的遗忘模型，研究了 “机器遗忘”（Machine unlearning）中的视觉数据遗忘问题，提出了一种高效的方法 Single Image Unlearning（SIU），通过对单个相关图像进行微调，实现对概念的可视化识别的遗忘。实验证明 SIU 显著优于现有方法，并能够避免入侵式成员推断攻击和越狱攻击。