Convolutional Neural Networks (CNN) are widely used to face challenging tasks
like speech recognition, natural language processing or computer vision. As CNN
architectures get larger and more complex, their computational requirements
increase, incurring significant energetic costs and challenging their
deployment on resource-restricted devices. In this paper, we propose Optimizing
Convolutional Neural Network Architecture (OCNNA), a novel CNN optimization and
construction method based on pruning and knowledge distillation designed to
establish the importance of convolutional layers. The proposal has been
evaluated though a thorough empirical study including the best known datasets
(CIFAR-10, CIFAR-100 and Imagenet) and CNN architectures (VGG-16, ResNet-50,
DenseNet-40 and MobileNet), setting Accuracy Drop and Remaining Parameters
Ratio as objective metrics to compare the performance of OCNNA against the
other state-of-art approaches. Our method has been compared with more than 20
convolutional neural network simplification algorithms obtaining outstanding
results. As a result, OCNNA is a competitive CNN constructing method which
could ease the deployment of neural networks into IoT or resource-limited
devices.

卷积神经网络（CNN）在面对声音识别、自然语言处理或计算机视觉等具有挑战性的任务时被广泛使用。本文提出了一种新颖的 CNN 优化和构建方法，基于剪枝和知识蒸馏，旨在确定卷积层的重要性，以减少计算要求，并在资源受限设备上部署。经过彻底的实证研究，包括最佳数据集（CIFAR-10、CIFAR-100 和 Imagenet）和 CNN 架构（VGG-16、ResNet-50、DenseNet-40 和 MobileNet），我们评估了该提议，并将准确度损失和剩余参数比率作为客观指标来比较 OCNNA 与其他最先进的方法的性能。与其他 20 多种卷积神经网络简化算法相比，我们的方法取得了出色的结果。因此，OCNNA 是一种竞争性的 CNN 构建方法，可以简化神经网络部署到物联网或资源有限设备的过程。

卷积神经网络架构优化

Optimizing Convolutional Neural Network Architecture

Quantization of deep neural networks is a promising approach that reduces the
inference cost, making it feasible to run deep networks on resource-restricted
devices. Inspired by existing methods, we propose a new framework to learn the
quantization intervals (discrete values) using the knowledge of the network's
weight and activation distributions, i.e., standard deviation. Furthermore, we
propose a novel base-2 logarithmic quantization scheme to quantize weights to
power-of-two discrete values. Our proposed scheme allows us to replace
resource-hungry high-precision multipliers with simple shift-add operations.
According to our evaluations, our method outperforms existing work on CIFAR10
and ImageNet datasets and even achieves better accuracy performance with 3-bit
weights and activations when compared to the full-precision models. Moreover,
our scheme simultaneously prunes the network's parameters and allows us to
flexibly adjust the pruning ratio during the quantization process.

本文提出了基于权重和激活分布的标准差的量化区间学习框架，同时引入了一种基于 base-2 对数的量化模式，该模式不仅证明了较好的精度表现，还能进行参数剪枝。