Channel pruning is one of the predominant approaches for accelerating deep
neural networks. Most existing pruning methods either train from scratch with a
sparsity inducing term such as group lasso, or prune redundant channels in a
pretrained network and then fine tune the network. Both strategies suffer from
some limitations: the use of group lasso is computationally expensive,
difficult to converge and often suffers from worse behavior due to the
regularization bias. The methods that start with a pretrained network either
prune channels uniformly across the layers or prune channels based on the basic
statistics of the network parameters. These approaches either ignore the fact
that some CNN layers are more redundant than others or fail to adequately
identify the level of redundancy in different layers. In this work, we
investigate a simple-yet-effective method for pruning channels based on a
computationally light-weight yet effective data driven optimization step that
discovers the necessary width per layer. Experiments conducted on ILSVRC-$12$
confirm effectiveness of our approach. With non-uniform pruning across the
layers on ResNet-$50$, we are able to match the FLOP reduction of
state-of-the-art channel pruning results while achieving a $0.98\%$ higher
accuracy. Further, we show that our pruned ResNet-$50$ network outperforms
ResNet-$34$ and ResNet-$18$ networks, and that our pruned ResNet-$101$
outperforms ResNet-$50$.

本文提出了一种简单而有效的基于数据驱动优化的频道修剪方法，该方法通过每层发现必要的宽度，以非均匀修剪 ResNet-50 等深度神经网络，实现了与现有方法相同的 FLOP 降低率，同时提高了 0.98％的准确度，并优于其他深度神经网络如 ResNet-34 和 ResNet-18。

PruneNet: 全局重要性通道剪枝

PruneNet: Channel Pruning via Global Importance

Ensembles of decision trees are a useful tool for obtaining for obtaining
flexible estimates of regression functions. Examples of these methods include
gradient boosted decision trees, random forests, and Bayesian CART. Two
potential shortcomings of tree ensembles are their lack of smoothness and
vulnerability to the curse of dimensionality. We show that these issues can be
overcome by instead considering sparsity inducing soft decision trees in which
the decisions are treated as probabilistic. We implement this in the context of
the Bayesian additive regression trees framework, and illustrate its promising
performance through testing on benchmark datasets. We provide strong
theoretical support for our methodology by showing that the posterior
distribution concentrates at the minimax rate (up-to a logarithmic factor) for
sparse functions and functions with additive structures in the high-dimensional
regime where the dimensionality of the covariate space is allowed to grow near
exponentially in the sample size. Our method also adapts to the unknown
smoothness and sparsity levels, and can be implemented by making minimal
modifications to existing BART algorithms.

使用稀疏感应的 soft decision trees 取代传统的决策树集成方法，可以克服它们的光滑性差和受维度灾难的困扰。在 Bayesian additive regression trees 框架下实现该方法，理论上得到强大的支持，并在基准数据集上表现出良好的性能。