To reduce the computation cost and the energy consumption in large language
models (LLM), skimming-based acceleration dynamically drops unimportant tokens
of the input sequence progressively along layers of the LLM while preserving
the tokens of semantic importance. However, our work for the first time reveals
the acceleration may be vulnerable to Denial-of-Service (DoS) attacks. In this
paper, we propose No-Skim, a general framework to help the owners of
skimming-based LLM to understand and measure the robustness of their
acceleration scheme. Specifically, our framework searches minimal and
unnoticeable perturbations at character-level and token-level to generate
adversarial inputs that sufficiently increase the remaining token ratio, thus
increasing the computation cost and energy consumption. We systematically
evaluate the vulnerability of the skimming acceleration in various LLM
architectures including BERT and RoBERTa on the GLUE benchmark. In the worst
case, the perturbation found by No-Skim substantially increases the running
cost of LLM by over 145% on average. Moreover, No-Skim extends the evaluation
framework to various scenarios, making the evaluation conductible with
different level of knowledge.

为了降低大型语言模型中的运算成本和能源消耗，本论文提出了一种名为 No-Skim 的通用框架，以帮助基于打桩的语言模型所有者了解和度量其加速方案的强健性，并通过搜索最小且不易察觉的字符级和标记级失真生成对抗性输入，从而增加剩余标记比率，进而增加运算成本和能源消耗。在各种 LLM 架构上对 skimming 加速的脆弱性进行了系统评估，包括 BERT 和 RoBERTa，并在 GLUE 基准测试中找到的失真在最坏情况下能够使 LLM 的运行成本平均增加超过 145％。此外，No-Skim 扩展了评估框架，以用于不同程度的知识。