We present ELSA, a practical solution for creating deep networks that can
easily be deployed at different levels of sparsity. The core idea is to embed
one or more sparse networks within a single dense network as a proper subset of
the weights. At prediction time, any sparse model can be extracted effortlessly
simply be zeroing out weights according to a predefined mask. ELSA is simple,
powerful and highly flexible. It can use essentially any existing technique for
network sparsification and network training. In particular, it does not
restrict the loss function, architecture or the optimization technique. Our
experiments show that ELSA's advantages of flexible deployment comes with no or
just a negligible reduction in prediction quality compared to the standard way
of using multiple sparse networks that are trained and stored independently.

ELSA 是一个实用的解决方案，可以在不同稀疏级别上轻松部署的深度网络。在预测时，只需根据预定义的掩码将权重清零，即可轻松提取任何稀疏模型，ELSA 具有简单、强大和高度灵活的特点，可以使用几乎任何现有的网络稀疏化和网络训练技术。实验证明，与独立训练和存储的多个稀疏网络相比，ELSA 在灵活部署方面的优势几乎不会或只会带来可忽略的预测质量下降。

ELSA：无额外开销的稀疏网络部署中的部分权重冻结

ELSA: Partial Weight Freezing for Overhead-Free Sparse Network  Deployment

We present any-precision deep neural networks (DNNs), which are trained with
a new method that allows the learned DNNs to be flexible in numerical precision
during inference. The same model in runtime can be flexibly and directly set to
different bit-widths, by truncating the least significant bits, to support
dynamic speed and accuracy trade-off. When all layers are set to low-bits, we
show that the model achieved accuracy comparable to dedicated models trained at
the same precision. This nice property facilitates flexible deployment of deep
learning models in real-world applications, where in practice trade-offs
between model accuracy and runtime efficiency are often sought. Previous
literature presents solutions to train models at each individual fixed
efficiency/accuracy trade-off point. But how to produce a model flexible in
runtime precision is largely unexplored. When the demand of efficiency/accuracy
trade-off varies from time to time or even dynamically changes in runtime, it
is infeasible to re-train models accordingly, and the storage budget may forbid
keeping multiple models. Our proposed framework achieves this flexibility
without performance degradation. More importantly, we demonstrate that this
achievement is agnostic to model architectures and applicable to multiple
vision tasks. Our code is released at
this https URL

提出了一种新方法训练任意精度深度神经网络（DNNs），可以在推断时灵活地设置数字精度，从而支持动态速度和准确度的折衷，并且该方法不会降低性能，该方法适用于多个视觉任务和模型结构