In-context learning (ICL) has emerged as a powerful paradigm leveraging LLMs
for specific tasks by utilizing labeled examples as demonstrations in the
precondition prompts. Despite its promising performance, ICL suffers from
instability with the choice and arrangement of examples. Additionally, crafted
adversarial attacks pose a notable threat to the robustness of ICL. However,
existing attacks are either easy to detect, rely on external models, or lack
specificity towards ICL. To address these issues, this work introduces a novel
transferable attack for ICL, aiming to hijack LLMs to generate the targeted
response. The proposed LLM hijacking attack leverages a gradient-based prompt
search method to learn and append imperceptible adversarial suffixes to the
in-context demonstrations. Extensive experimental results on various tasks and
datasets demonstrate the effectiveness of our LLM hijacking attack, resulting
in a distracted attention towards adversarial tokens, consequently leading to
the targeted unwanted outputs.

通过引入一种新的对上下文学习的颠覆性攻击方法，本文展示了一种能够利用 LMLs 来生成针对性响应的方法，并通过对各种任务和数据集的广泛实验结果证明了其有效性。

通过对抗性上下文学习劫持大型语言模型

Hijacking Large Language Models via Adversarial In-Context Learning

The vulnerability of deep neural networks to adversarial examples has drawn
tremendous attention from the community. Three approaches, optimizing standard
objective functions, exploiting attention maps, and smoothing decision
surfaces, are commonly used to craft adversarial examples. By tightly
integrating the three approaches, we propose a new and simple algorithm named
Transferable Attack based on Integrated Gradients (TAIG) in this paper, which
can find highly transferable adversarial examples for black-box attacks. Unlike
previous methods using multiple computational terms or combining with other
methods, TAIG integrates the three approaches into one single term. Two
versions of TAIG that compute their integrated gradients on a straight-line
path and a random piecewise linear path are studied. Both versions offer strong
transferability and can seamlessly work together with the previous methods.
Experimental results demonstrate that TAIG outperforms the state-of-the-art
methods. The code will available at this https URL

通过将标准客观函数优化、利用注意力图和平滑决策表面三种方法紧密整合，我们在本文中提出了一种新的简单算法 TAIG (Transferable Attack based on Integrated Gradients)，它可以为黑盒攻击寻找高度可转移的敌对样本，两个版本的 TAIG 都计算了沿着直线或随机分段线性路径的积分梯度，均具有很强的可转移性，并且可以与以前的方法无缝协同工作。实验结果表明，TAIG 优于现有的最先进方法 。

基于集成梯度的可迁移对抗攻击

Transferable Adversarial Attack based on Integrated Gradients

Existing works have identified the limitation of top-$1$ attack success rate
(ASR) as a metric to evaluate the attack strength but exclusively investigated
it in the white-box setting, while our work extends it to a more practical
black-box setting: transferable attack. It is widely reported that stronger
I-FGSM transfers worse than simple FGSM, leading to a popular belief that
transferability is at odds with the white-box attack strength. Our work
challenges this belief with empirical finding that stronger attack actually
transfers better for the general top-$k$ ASR indicated by the interest class
rank (ICR) after attack. For increasing the attack strength, with an intuitive
interpretation of the logit gradient from the geometric perspective, we
identify that the weakness of the commonly used losses lie in prioritizing the
speed to fool the network instead of maximizing its strength. To this end, we
propose a new normalized CE loss that guides the logit to be updated in the
direction of implicitly maximizing its rank distance from the ground-truth
class. Extensive results in various settings have verified that our proposed
new loss is simple yet effective for top-$k$ attack. Code is available at:
https://bit.ly/3uCiomP

本文探讨了在实际应用中的黑盒情况下，提出了对攻击强度的评估。作为摘要，我们从几何角度出发，提出了新的 CE 损失函数，该函数针对攻击强度和 ICR 等方面进行了有效的实现和验证。

研究前 $k$ 名白盒和可转移黑盒攻击

Investigating Top-$k$ White-Box and Transferable Black-box Attack

Deep Neural Networks have been found vulnerable re-cently. A kind of
well-designed inputs, which called adver-sarial examples, can lead the networks
to make incorrectpredictions. Depending on the different scenarios, goalsand
capabilities, the difficulties of the attacks are different.For example, a
targeted attack is more difficult than a non-targeted attack, a universal
attack is more difficult than anon-universal attack, a transferable attack is
more difficultthan a nontransferable one. The question is: Is there existan
attack that can meet all these requirements? In this pa-per, we answer this
question by producing a kind of attacksunder these conditions. We learn a
universal mapping tomap the sources to the adversarial examples. These
exam-ples can fool classification networks to classify all of theminto one
targeted class, and also have strong transferability.Our code is released at:
xxxxx.

本文使用一种名为对抗性样本的设计，产生一个可以同时攻击深度神经网络的通用攻击，包括目标攻击，通用攻击以及可转移攻击。