Large Language Models (LLMs) have demonstrated impressive performance on a
range of Natural Language Processing (NLP) tasks. Unfortunately, the immense
amount of computations and memory accesses required for LLM training makes them
prohibitively expensive in terms of hardware cost, and thus challenging to
deploy in use cases such as on-device learning. In this paper, motivated by the
observation that LLM training is memory-bound, we propose a novel dynamic
quantization strategy, termed Dynamic Stashing Quantization (DSQ), that puts a
special focus on reducing the memory operations, but also enjoys the other
benefits of low precision training, such as the reduced arithmetic cost. We
conduct a thorough study on two translation tasks (trained-from-scratch) and
three classification tasks (fine-tuning). DSQ reduces the amount of arithmetic
operations by $20.95\times$ and the number of DRAM operations by $2.55\times$
on IWSLT17 compared to the standard 16-bit fixed-point, which is widely used in
on-device learning.

本文提出一种新颖的动态存储量化策略，Dynamic Stashing Quantization（DSQ），旨在减少内存操作，并享受低精度训练的其他好处，比如减少运算成本，通过在两个翻译任务和三个分类任务上的研究表明，DSQ 在 IWSLT17 上将算术操作的数量降低了 20.95 倍，并将 DRAM 操作数量降低了 2.55 倍，这为大语言模型在设备学习中的部署提供了解决方案。