This paper starts with a simple lossless ~1.5:1 compression algorithm for the
weights of the Large Language Model (LLM) Llama2 7B [1] that can be implemented
in ~200 LUTs in AMD FPGAs, processing over 800 million bfloat16 numbers per
second. This framework is then extended to variable precision, variable range,
compressed numerical data types that are a user defined super set of both
floats and posits [2]. The paper then discusses a simple hardware
implementation of such format based on ANS (Asymmetrical Numeral Systems) [3]
that acts as a bridge between this flexible data format and a computational
engine while, at the same time, achieving bandwidth reduction. An example of a
token factory using weight compression and sharing is also given.

这篇论文介绍了一种简单的无损压缩算法，用于处理大型语言模型的权重，可在 AMD FPGAs 中实现，并能每秒处理超过 8 亿个 bfloat16 数字。该算法被扩展应用于可变精度、可变范围的数值数据类型，并讨论了基于 ANS（非对称数系统）的硬件实现方法，该方法在实现数据格式的灵活性和计算引擎之间起到了桥梁作用，并实现了带宽的降低。同时，还给出了使用权重压缩和共享的令牌生成器的示例。

从 Llama2 7B 权重的无损（~1.5:1）压缩算法到 CNNs 和 LLMs 的可变精度、可变范围、压缩数值数据类型

From a Lossless (~1.5:1) Compression Algorithm for Llama2 7B Weights to  Variable Precision, Variable Range, Compressed Numeric Data Types for CNNs  and LLMs