Large Language Models (LLMs) have exhibited remarkable proficiency across a
wide array of NLP tasks. However, the escalation in model size also engenders
substantial deployment costs. While few efforts have explored model pruning
techniques to reduce the size of LLMs, they mainly center on general or
task-specific weights. This leads to suboptimal performance due to lacking
specificity on the target domain or generality on different tasks when applied
to domain-specific challenges. This work introduces an innovative unstructured
dual-pruning methodology, D-Pruner, for domain-specific compression on LLM. It
extracts a compressed, domain-specific, and task-agnostic LLM by identifying
LLM weights that are pivotal for general capabilities, like linguistic
capability and multi-task solving, and domain-specific knowledge. More
specifically, we first assess general weight importance by quantifying the
error incurred upon their removal with the help of an open-domain calibration
dataset. Then, we utilize this general weight importance to refine the training
loss, so that it preserves generality when fitting into a specific domain.
Moreover, by efficiently approximating weight importance with the refined
training loss on a domain-specific calibration dataset, we obtain a pruned
model emphasizing generality and specificity. Our comprehensive experiments
across various tasks in healthcare and legal domains show the effectiveness of
D-Pruner in domain-specific compression. Our code is available at
this https URL

通过识别关键权重，提出了一种创新的针对特定领域的压缩方法 D-Pruner，通过提取关键权重，实现 Compressed, domain-specific 和 task-agnostic 的 LLM，具有广泛适用性和特定性。

领域特定的 LLM 提取器剪枝

Pruning as a Domain-specific LLM Extractor

Object detectors used in autonomous vehicles can have high memory and
computational overheads. In this paper, we introduce a novel semi-structured
pruning framework called R-TOSS that overcomes the shortcomings of
state-of-the-art model pruning techniques. Experimental results on the
JetsonTX2 show that R-TOSS has a compression rate of 4.4x on the YOLOv5 object
detector with a 2.15x speedup in inference time and 57.01% decrease in energy
usage. R-TOSS also enables 2.89x compression on RetinaNet with a 1.86x speedup
in inference time and 56.31% decrease in energy usage. We also demonstrate
significant improvements compared to various state-of-the-art pruning
techniques.

本文提出一种名为 R-TOSS 的新型半结构化剪枝框架，可以在不降低检测性能的情况下大幅减少对象检测器的内存使用和计算时间，可以实现可观的压缩和能耗降低并且优于现有剪枝技术。