Quantization techniques commonly reduce the inference costs of neural
networks by restricting the precision of weights and activations. Recent
studies show that also reducing the precision of the accumulator can further
improve hardware efficiency at the risk of numerical overflow, which introduces
arithmetic errors that can degrade model accuracy. To avoid numerical overflow
while maintaining accuracy, recent work proposed accumulator-aware quantization
(A2Q), a quantization-aware training method that constrains model weights
during training to safely use a target accumulator bit width during inference.
Although this shows promise, we demonstrate that A2Q relies on an overly
restrictive constraint and a sub-optimal weight initialization strategy that
each introduce superfluous quantization error. To address these shortcomings,
we introduce: (1) an improved bound that alleviates accumulator constraints
without compromising overflow avoidance; and (2) a new strategy for
initializing quantized weights from pre-trained floating-point checkpoints. We
combine these contributions with weight normalization to introduce A2Q+. We
support our analysis with experiments that show A2Q+ significantly improves the
trade-off between accumulator bit width and model accuracy and characterize new
trade-offs that arise as a consequence of accumulator constraints.

通过限制权重和激活函数的精度，量化技术通常降低神经网络推理成本。最近的研究表明，降低累加器的精度可以进一步提高硬件效率，但存在数值溢出的风险，这会导致算术错误并降低模型的准确性。为了避免数值溢出并保持准确性，最新的工作提出了一种称为累加器感知量化（A2Q）的量化感知训练方法，在训练期间约束模型权重以在推理过程中安全地使用目标累加器位宽。尽管这显示出了潜力，但我们证明 A2Q 依赖于过于严格的约束和亚优化的权重初始化策略，每个都引入了多余的量化误差。为了解决这些缺点，我们引入了两个改进：（1）一种改进的上界，缓解了累加器约束而不损害溢出避免；（2）一种从预训练的浮点检查点初始化量化权重的新策略。我们将这些贡献与权重归一化结合起来，引入 A2Q+。我们通过实验证实，A2Q + 显著改善了累加器位宽和模型准确性之间的权衡，并表征了累加器约束引起的新的权衡。