Currently, deep neural networks are deployed on low-power portable devices by
first training a full-precision model using powerful hardware, and then
deriving a corresponding low-precision model for efficient inference on such
systems. However, training models directly with coarsely quantized weights is a
key step towards learning on embedded platforms that have limited computing
resources, memory capacity, and power consumption. Numerous recent publications
have studied methods for training quantized networks, but these studies have
mostly been empirical. In this work, we investigate training methods for
quantized neural networks from a theoretical viewpoint. We first explore
accuracy guarantees for training methods under convexity assumptions. We then
look at the behavior of these algorithms for non-convex problems, and show that
training algorithms that exploit high-precision representations have an
important greedy search phase that purely quantized training methods lack,
which explains the difficulty of training using low-precision arithmetic.

本篇论文研究了深度神经网络的低精度模型的训练方法，探讨了在小型设备上训练深度神经网络的难点，提出了一种理论视角下的量化训练方法，并探索了非凸问题下的算法行为模型。