Large Language Models (LLMs) have demonstrated remarkable capabilities in
various domains, but their vulnerability to trojan or backdoor attacks poses
significant security risks. This paper explores the challenges and insights
gained from the Trojan Detection Competition 2023 (TDC2023), which focused on
identifying and evaluating trojan attacks on LLMs. We investigate the
difficulty of distinguishing between intended and unintended triggers, as well
as the feasibility of reverse engineering trojans in real-world scenarios. Our
comparative analysis of various trojan detection methods reveals that achieving
high Recall scores is significantly more challenging than obtaining high
Reverse-Engineering Attack Success Rate (REASR) scores. The top-performing
methods in the competition achieved Recall scores around 0.16, comparable to a
simple baseline of randomly sampling sentences from a distribution similar to
the given training prefixes. This finding raises questions about the
detectability and recoverability of trojans inserted into the model, given only
the harmful targets. Despite the inability to fully solve the problem, the
competition has led to interesting observations about the viability of trojan
detection and improved techniques for optimizing LLM input prompts. The
phenomenon of unintended triggers and the difficulty in distinguishing them
from intended triggers highlights the need for further research into the
robustness and interpretability of LLMs. The TDC2023 has provided valuable
insights into the challenges and opportunities associated with trojan detection
in LLMs, laying the groundwork for future research in this area to ensure their
safety and reliability in real-world applications.

大语言模型（LLMs）在各个领域展示出了显著的能力，但它们对木马或后门攻击的脆弱性带来了重大的安全风险。本研究探讨了通过特洛伊检测竞赛 2023（TDC2023）获得的挑战和见解，该竞赛的重点是识别和评估对 LLMs 的特洛伊攻击。我们研究了区分有意和无意触发器之间的困难，以及在实际情况中逆向工程特洛伊的可行性。我们对各种特洛伊检测方法进行了比较分析，发现实现高召回率比获得高逆向工程攻击成功率（REASR）更具挑战性。竞赛中表现最佳的方法实现了约 0.16 的召回率，与从与训练前缀类似的分布中随机抽取句子的简单基线相当。这一发现提出了关于仅给定有害目标的情况下模型中插入的特洛伊的可检测性和恢复性的问题。尽管无法完全解决问题，但该竞赛揭示了关于特洛伊检测的可行性和改进 LLM 输入提示技术的有趣观察。无意触发器的现象及其与有意触发器的区分困难突显了对 LLMs 的鲁棒性和可解释性进行进一步研究的需求。TDC2023 对于特洛伊检测在 LLMs 中的挑战和机遇提供了宝贵的见解，为今后在这一领域确保其在实际应用中的安全性和可靠性奠定了基础。

大型语言模型中的特洛伊木马检测：来自特洛伊木马检测挑战的见解

Trojan Detection in Large Language Models: Insights from The Trojan  Detection Challenge

We empirically study a simple layer-pruning strategy for popular families of
open-weight pretrained LLMs, finding minimal degradation of performance on
different question-answering benchmarks until after a large fraction (up to
half) of the layers are removed. To prune these models, we identify the optimal
block of layers to prune by considering similarity across layers; then, to
"heal" the damage, we perform a small amount of finetuning. In particular, we
use parameter-efficient finetuning (PEFT) methods, specifically quantization
and Low Rank Adapters (QLoRA), such that each of our experiments can be
performed on a single A100 GPU. From a practical perspective, these results
suggest that layer pruning methods can complement other PEFT strategies to
further reduce computational resources of finetuning on the one hand, and can
improve the memory and latency of inference on the other hand. From a
scientific perspective, the robustness of these LLMs to the deletion of layers
implies either that current pretraining methods are not properly leveraging the
parameters in the deeper layers of the network or that the shallow layers play
a critical role in storing knowledge.

通过对热门的开放式权重预训练 LLMs 进行实证研究，我们发现在移除大部分（最多一半）层之前，其在不同的问答基准测试中仅出现轻微性能下降；因此，层剪枝方法可以在减少计算资源的同时提高推理的内存和延迟，并暗示当前的预训练方法未充分利用网络较深层的参数，或者浅层起到了关键的存储知识的作用。