Machine Unlearning is the process of removing specific training data samples
and their corresponding effects from an already trained model. It has
significant practical benefits, such as purging private, inaccurate, or
outdated information from trained models without the need for complete
re-training. Unlearning within a multimodal setting presents unique challenges
due to the intrinsic dependencies between different data modalities and the
expensive cost of training on large multimodal datasets and architectures.
Current approaches to machine unlearning have not fully addressed these
challenges. To bridge this gap, we introduce MMUL, a machine unlearning
approach specifically designed for multimodal data and models. MMUL formulates
the multimodal unlearning task by focusing on three key properties: (a):
modality decoupling, which effectively decouples the association between
individual unimodal data points within multimodal inputs marked for deletion,
rendering them as unrelated data points within the model's context, (b):
unimodal knowledge retention, which retains the unimodal representation
capability of the model post-unlearning, and (c): multimodal knowledge
retention, which retains the multimodal representation capability of the model
post-unlearning. MMUL is efficient to train and is not constrained by the
requirement of using a strongly convex loss. Experiments on two multimodal
models and four multimodal benchmark datasets, including vision-language and
graph-language datasets, show that MMUL outperforms existing baselines, gaining
an average improvement of +17.6 points against the best-performing unimodal
baseline in distinguishing between deleted and remaining data. In addition,
MMUL can largely maintain pre-existing knowledge of the original model post
unlearning, with a performance gap of only 0.3 points compared to retraining a
new model from scratch.

Machine Unlearning（机器遗忘）是从已经训练好的模型中删除特定的训练数据样本及其相应的影响，其具有重要的实际价值。本文提出了 MMUL，一种专门设计用于多模态数据和模型的机器遗忘方法，通过关注三个关键性质来制定多模态遗忘任务：（a）解耦模态，（b）保留单模态知识，（c）保留多模态知识。实验证明，MMUL 在区分已删除和剩余数据方面优于现有的基线方法，并且在遗忘后能够大部分保持原始模型的预先存在的知识。