This study considers the attack on reinforcement learning agents where the
adversary aims to control the victim's behavior as specified by the adversary
by adding adversarial modifications to the victim's state observation. While
some attack methods reported success in manipulating the victim agent's
behavior, these methods often rely on environment-specific heuristics. In
addition, all existing attack methods require white-box access to the victim's
policy. In this study, we propose a novel method for manipulating the victim
agent in the black-box (i.e., the adversary is allowed to observe the victim's
state and action only) and no-box (i.e., the adversary is allowed to observe
the victim's state only) setting without requiring environment-specific
heuristics. Our attack method is formulated as a bi-level optimization problem
that is reduced to a distribution matching problem and can be solved by an
existing imitation learning algorithm in the black-box and no-box settings.
Empirical evaluations on several reinforcement learning benchmarks show that
our proposed method has superior attack performance to baselines.

研究考虑攻击强化学习代理，其中对手通过对受害者的状态观察添加敌对修改来控制受害者的行为，提出了一种在黑盒和无盒设置中操纵受害者代理的新方法，无需特定环境启发式，并通过分布匹配问题解决现有的模仿学习算法，实证评估表明该方法攻击性能优于基准方法。

针对受攻击方策略访问有限的强化学习的行为目标攻击

Behavior-Targeted Attack on Reinforcement Learning with Limited Access  to Victim's Policy

Contextual bandit algorithms are applied in a wide range of domains, from
advertising to recommender systems, from clinical trials to education. In many
of these domains, malicious agents may have incentives to attack the bandit
algorithm to induce it to perform a desired behavior. For instance, an
unscrupulous ad publisher may try to increase their own revenue at the expense
of the advertisers; a seller may want to increase the exposure of their
products, or thwart a competitor's advertising campaign. In this paper, we
study several attack scenarios and show that a malicious agent can force a
linear contextual bandit algorithm to pull any desired arm $T - o(T)$ times
over a horizon of $T$ steps, while applying adversarial modifications to either
rewards or contexts that only grow logarithmically as $O(\log T)$. We also
investigate the case when a malicious agent is interested in affecting the
behavior of the bandit algorithm in a single context (e.g., a specific user).
We first provide sufficient conditions for the feasibility of the attack and we
then propose an efficient algorithm to perform the attack. We validate our
theoretical results on experiments performed on both synthetic and real-world
datasets.

本文研究了恶意代理如何在 linear contextual bandit algorithm 上执行攻击，并提出了一种有效的算法来执行此攻击。