In this paper, we devise a novel lightweight Graph Convolutional Network
(GCN) design dubbed as Multi-Rate Magnitude Pruning (MRMP) that jointly trains
network topology and weights. Our method is variational and proceeds by
aligning the weight distribution of the learned networks with an a priori
distribution. In the one hand, this allows implementing any fixed pruning rate,
and also enhancing the generalization performances of the designed lightweight
GCNs. In the other hand, MRMP achieves a joint training of multiple GCNs, on
top of shared weights, in order to extrapolate accurate networks at any
targeted pruning rate without retraining their weights. Extensive experiments
conducted on the challenging task of skeleton-based recognition show a
substantial gain of our lightweight GCNs particularly at very high pruning
regimes.

我们提出了一种轻量级图卷积网络设计，称为 Multi-Rate Magnitude Pruning (MRMP)，它通过联合训练网络拓扑和权重来对网络进行训练。该方法是变分的，通过将学习网络的权重分布与先验分布对齐，实现了任何固定的剪枝率的具体操作，并增强了设计的轻量级 GCN 的泛化性能。此外，MRMP 通过共享权重在多个 GCN 之间进行联合训练，以在任何目标剪枝率上外推准确的网络，而无需重新训练它们的权重。对基于骨骼的识别这一具有挑战性的任务进行的大量实验表明，我们的轻量级 GCN 在非常高的剪枝程度下有显著的性能提高。

图卷积网络的一次性多速率剪枝

One-Shot Multi-Rate Pruning of Graph Convolutional Networks

Channel pruning can effectively reduce both computational cost and memory
footprint of the original network while keeping a comparable accuracy
performance. Though great success has been achieved in channel pruning for 2D
image-based convolutional networks (CNNs), existing works seldom extend the
channel pruning methods to 3D point-based neural networks (PNNs). Directly
implementing the 2D CNN channel pruning methods to PNNs undermine the
performance of PNNs because of the different representations of 2D images and
3D point clouds as well as the network architecture disparity. In this paper,
we proposed CP$^3$, which is a Channel Pruning Plug-in for Point-based network.
CP$^3$ is elaborately designed to leverage the characteristics of point clouds
and PNNs in order to enable 2D channel pruning methods for PNNs. Specifically,
it presents a coordinate-enhanced channel importance metric to reflect the
correlation between dimensional information and individual channel features,
and it recycles the discarded points in PNN's sampling process and reconsiders
their potentially-exclusive information to enhance the robustness of channel
pruning. Experiments on various PNN architectures show that CP$^3$ constantly
improves state-of-the-art 2D CNN pruning approaches on different point cloud
tasks. For instance, our compressed PointNeXt-S on ScanObjectNN achieves an
accuracy of 88.52% with a pruning rate of 57.8%, outperforming the baseline
pruning methods with an accuracy gain of 1.94%.

本文提出了一个针对 3D 点云的 Channel Pruning 插件 CP^3，通过坐标增强特征重要性度量和回收舍弃点云信息来提高通道裁剪的鲁棒性，实验证明 CP^3 在不同点云任务上均比基线方法性能更好。

CP$^3$: 面向基于点云的网络的通道剪枝插件

CP$^3$: Channel Pruning Plug-in for Point-based Networks

The exponential growth in numbers of parameters of neural networks over the
past years has been accompanied by an increase in performance across several
fields. However, due to their sheer size, the networks not only became
difficult to interpret but also problematic to train and use in real-world
applications, since hardware requirements increased accordingly. Tackling both
issues, we present a novel approach that either drops a neural network's
initial weights or inverts their respective sign. Put simply, a network is
trained by weight selection and inversion without changing their absolute
values. Our contribution extends previous work on masking by additionally
sign-inverting the initial weights and follows the findings of the Lottery
Ticket Hypothesis. Through this extension and adaptations of initialization
methods, we achieve a pruning rate of up to 99%, while still matching or
exceeding the performance of various baseline and previous models. Our approach
has two main advantages. First, and most notable, signed Supermask models
drastically simplify a model's structure, while still performing well on given
tasks. Second, by reducing the neural network to its very foundation, we gain
insights into which weights matter for performance. The code is available on
GitHub.

本文提出了一种新颖的方法，通过权重选择和反转来训练神经网络，达到高达 99% 的剪枝率，而仍然可以匹配或超过各种基线和以前的模型的性能。