Machine unlearning has garnered significant attention due to its ability to
selectively erase knowledge obtained from specific training data samples in an
already trained machine learning model. This capability enables data holders to
adhere strictly to data protection regulations. However, existing unlearning
techniques face practical constraints, often causing performance degradation,
demanding brief fine-tuning post unlearning, and requiring significant storage.
In response, this paper introduces a novel class of machine unlearning
algorithms. First method is partial amnesiac unlearning, integration of
layer-wise pruning with amnesiac unlearning. In this method, updates made to
the model during training are pruned and stored, subsequently used to forget
specific data from trained model. The second method assimilates layer-wise
partial-updates into label-flipping and optimization-based unlearning to
mitigate the adverse effects of data deletion on model efficacy. Through a
detailed experimental evaluation, we showcase the effectiveness of proposed
unlearning methods. Experimental results highlight that the partial amnesiac
unlearning not only preserves model efficacy but also eliminates the necessity
for brief post fine-tuning, unlike conventional amnesiac unlearning. Moreover,
employing layer-wise partial updates in label-flipping and optimization-based
unlearning techniques demonstrates superiority in preserving model efficacy
compared to their naive counterparts.

该文献介绍了一类新的机器遗忘算法，其中包括部分失忆式遗忘和逐层剪枝。通过详细的实验评估，展示了所提出的遗忘方法的有效性，部分失忆式遗忘不仅保持了模型效果，还消除了简短预训练的必要性，而逐层部分更新在标签翻转和基于优化的遗忘技术中显示了较于其原始方法更优的模型效果保留能力。

通过逐层部分模型遗忘实现的训练模型高效知识删除

Efficient Knowledge Deletion from Trained Models through Layer-wise  Partial Machine Unlearning

Large Vision-Language Models (LVLMs) can understand the world comprehensively
by integrating rich information from different modalities, achieving remarkable
performance improvements on various multimodal downstream tasks. However,
deploying LVLMs is often problematic due to their massive computational/energy
costs and carbon consumption. Such issues make it infeasible to adopt
conventional iterative global pruning, which is costly due to computing the
Hessian matrix of the entire large model for sparsification. Alternatively,
several studies have recently proposed layer-wise pruning approaches to avoid
the expensive computation of global pruning and efficiently compress model
weights according to their importance within a layer. However, these methods
often suffer from suboptimal model compression due to their lack of a global
perspective. To address this limitation in recent efficient pruning methods for
large models, we propose Efficient Coarse-to-Fine Layer-Wise Pruning (ECoFLaP),
a two-stage coarse-to-fine weight pruning approach for LVLMs. We first
determine the sparsity ratios of different layers or blocks by leveraging the
global importance score, which is efficiently computed based on the
zeroth-order approximation of the global model gradients. Then, the multimodal
model performs local layer-wise unstructured weight pruning based on
globally-informed sparsity ratios. We validate our proposed method across
various multimodal and unimodal models and datasets, demonstrating significant
performance improvements over prevalent pruning techniques in the high-sparsity
regime.

通过 Efficient Coarse-to-Fine Layer-Wise Pruning 方法，LVLMs 的大规模视觉 - 语言模型能够以两阶段粗细权重修剪的方式进行模型压缩，从而在高稀疏度条件下取得显著性能改进。

ECoFLaP: 高效细粒度层次裁剪用于视觉 - 语言模型

ECoFLaP: Efficient Coarse-to-Fine Layer-Wise Pruning for Vision-Language  Models

How to develop slim and accurate deep neural networks has become crucial for
real- world applications, especially for those employed in embedded systems.
Though previous work along this research line has shown some promising results,
most existing methods either fail to significantly compress a well-trained deep
network or require a heavy retraining process for the pruned deep network to
re-boost its prediction performance. In this paper, we propose a new layer-wise
pruning method for deep neural networks. In our proposed method, parameters of
each individual layer are pruned independently based on second order
derivatives of a layer-wise error function with respect to the corresponding
parameters. We prove that the final prediction performance drop after pruning
is bounded by a linear combination of the reconstructed errors caused at each
layer. Therefore, there is a guarantee that one only needs to perform a light
retraining process on the pruned network to resume its original prediction
performance. We conduct extensive experiments on benchmark datasets to
demonstrate the effectiveness of our pruning method compared with several
state-of-the-art baseline methods.

本文提出了一种基于层次的剪枝方法对深度神经网络进行压缩，通过独立剪枝每个层次的参数可以保证压缩后的网络只需要进行轻量级的重新训练即可恢复预测能力，并在基准数据集上进行了广泛实验以证明其有效性。