Fine-tuning LLMs is crucial to enhancing their task-specific performance and
ensuring model behaviors are aligned with human preferences. Among various
fine-tuning methods, LoRA is popular for its efficiency and ease to use,
allowing end-users to easily post and adopt lightweight LoRA modules on
open-source platforms to tailor their model for different customization.
However, such a handy share-and-play setting opens up new attack surfaces, that
the attacker can render LoRA as an attacker, such as backdoor injection, and
widely distribute the adversarial LoRA to the community easily. This can result
in detrimental outcomes. Despite the huge potential risks of sharing LoRA
modules, this aspect however has not been fully explored. To fill the gap, in
this study we thoroughly investigate the attack opportunities enabled in the
growing share-and-play scenario. Specifically, we study how to inject backdoor
into the LoRA module and dive deeper into LoRA's infection mechanisms. We found
that training-free mechanism is possible in LoRA backdoor injection. We also
discover the impact of backdoor attacks with the presence of multiple LoRA
adaptions concurrently as well as LoRA based backdoor transferability. Our aim
is to raise awareness of the potential risks under the emerging share-and-play
scenario, so as to proactively prevent potential consequences caused by
LoRA-as-an-Attack. Warning: the paper contains potential offensive content
generated by models.

在这项研究中，我们深入研究了在不断增长的共享和使用情境中，如何将后门注入 LoRA 模块，并更深入地探索了 LoRA 的感染机制。我们发现，在 LoRA 后门注入中，无需进行训练即可实现机制。同时，我们还研究了当多个 LoRA 适应并存以及基于 LoRA 的后门传递能力时的后门攻击影响。我们的目标是提高人们对新兴共享和使用情境中潜在风险的意识，以便主动预防由 LoRA 作为攻击手段可能引发的潜在后果。