The efficient compression of large language models (LLMs) is becoming
increasingly popular. However, recovering the accuracy of compressed LLMs is
still a major challenge. Structural pruning with standard Low-Rank Adaptation
(LoRA) is a common technique in current LLM compression. In structural pruning,
the model architecture is modified unevenly, resulting in suboptimal
performance in various downstream tasks via standard LoRA with fixed rank. To
address this problem, we introduce RankAdaptor, an efficient fine-tuning method
with hierarchical dynamic rank scheduling for pruned LLMs. An end-to-end
automatic optimization flow is developed that utilizes a lightweight
performance model to determine the different ranks during fine-tuning.
Comprehensive experiments on popular benchmarks show that RankAdaptor
consistently outperforms standard LoRA with structural pruning over different
pruning settings. Without increasing the trainable parameters, RankAdaptor
further reduces the accuracy performance gap between the recovery of the pruned
model and the original model compared to standard LoRA.

采用 RankAdaptor 的分层动态秩调度方法，有效地微调剪枝的大型语言模型 (LLM)，在不增加训练参数的情况下，进一步减小剪枝模型恢复精度与原始模型之间的性能差距。