Backdoor attacks in reinforcement learning (RL) have previously employed
intense attack strategies to ensure attack success. However, these methods
suffer from high attack costs and increased detectability. In this work, we
propose a novel approach, BadRL, which focuses on conducting highly sparse
backdoor poisoning efforts during training and testing while maintaining
successful attacks. Our algorithm, BadRL, strategically chooses state
observations with high attack values to inject triggers during training and
testing, thereby reducing the chances of detection. In contrast to the previous
methods that utilize sample-agnostic trigger patterns, BadRL dynamically
generates distinct trigger patterns based on targeted state observations,
thereby enhancing its effectiveness. Theoretical analysis shows that the
targeted backdoor attack is always viable and remains stealthy under specific
assumptions. Empirical results on various classic RL tasks illustrate that
BadRL can substantially degrade the performance of a victim agent with minimal
poisoning efforts 0.003% of total training steps) during training and
infrequent attacks during testing.

我们提出了一种新方法 BadRL，它通过在训练和测试期间针对高攻击值的状态进行高度稀疏的后门毒化，从而有效地降低了被检测的几率，并且可以动态生成不同的触发模式来增强攻击的有效性。实验证明，BadRL 在多个典型强化学习任务中能够在训练期间以极小的毒化操作（占总训练步骤的 0.003%）显著降低受害智能体的性能，并在测试期间进行不频繁的攻击。