Due to their large size, generative Large Language Models (LLMs) require
significant computing and storage resources. This paper introduces a new
post-training quantization method, GPTQT, to reduce memory usage and enhance
processing speed by expressing the weight of LLM in 3bit/2bit. Practice has
shown that minimizing the quantization error of weights is ineffective, leading
to overfitting. Therefore, GPTQT employs a progressive two-step approach:
initially quantizing weights using Linear quantization to a relatively high
bit, followed by converting obtained int weight to lower bit binary coding. A
re-explore strategy is proposed to optimize initial scaling factor. During
inference, these steps are merged into pure binary coding, enabling efficient
computation. Testing across various models and datasets confirms GPTQT's
effectiveness. Compared to the strong 3-bit quantization baseline, GPTQT
further reduces perplexity by 4.01 on opt-66B and increases speed by 1.24 times
on opt-30b. The results on Llama2 show that GPTQT is currently the best binary
coding quantization method for such kind of LLMs.

该研究介绍了一种新的后训练量化方法 GPTQT，通过以 3 位 / 2 位表示 LLM 的权重，以减少内存使用并增强处理速度。经过测试，与强 3 位量化基准相比，GPTQT 在 opt-66B 上进一步降低了困惑度 4.01，并在 opt-30b 上提高了 1.24 倍的速度，说明 GPTQT 是目前针对此类 LLMs 的最佳二进制编码量化方法。

GPTQT：将大型语言模型量化两次以提高效率

GPTQT: Quantize Large Language Models Twice to Push the Efficiency

Person re-identification (ReID) aims at matching persons across different
views/scenes. In addition to accuracy, the matching efficiency has received
more and more attention because of demanding applications using large-scale
data. Several binary coding based methods have been proposed for efficient
ReID, which either learn projections to map high-dimensional features to
compact binary codes, or directly adopt deep neural networks by simply
inserting an additional fully-connected layer with tanh-like activations.
However, the former approach requires time-consuming hand-crafted feature
extraction and complicated (discrete) optimizations; the latter lacks the
necessary discriminative information greatly due to the straightforward
activation functions. In this paper, we propose a simple yet effective
framework for efficient ReID inspired by the recent advances in adversarial
learning. Specifically, instead of learning explicit projections or adding
fully-connected mapping layers, the proposed Adversarial Binary Coding (ABC)
framework guides the extraction of binary codes implicitly and effectively. The
discriminability of the extracted codes is further enhanced by equipping the
ABC with a deep triplet network for the ReID task. More importantly, the ABC
and triplet network are simultaneously optimized in an end-to-end manner.
Extensive experiments on three large-scale ReID benchmarks demonstrate the
superiority of our approach over the state-of-the-art methods.

本研究提出了一种简单而有效的基于对抗学习的二进制编码框架（ABC）用于高效实现人物重识别任务，通过配备深度三元组网络进一步增强所提取的二进制码的可辨别性，同时在端到端的方式下优化 ABC 和三元组网络，实验结果显示该方法在三个大规模的人物重识别基准测试中优于现有方法。