Deep reinforcement learning policies, which are integral to modern control
systems, represent valuable intellectual property. The development of these
policies demands considerable resources, such as domain expertise, simulation
fidelity, and real-world validation. These policies are potentially vulnerable
to model stealing attacks, which aim to replicate their functionality using
only black-box access. In this paper, we propose Stealthy Imitation, the first
attack designed to steal policies without access to the environment or
knowledge of the input range. This setup has not been considered by previous
model stealing methods. Lacking access to the victim's input states
distribution, Stealthy Imitation fits a reward model that allows to approximate
it. We show that the victim policy is harder to imitate when the distribution
of the attack queries matches that of the victim. We evaluate our approach
across diverse, high-dimensional control tasks and consistently outperform
prior data-free approaches adapted for policy stealing. Lastly, we propose a
countermeasure that significantly diminishes the effectiveness of the attack.

这篇论文介绍了一种名为 Stealthy Imitation 的攻击方法，旨在不接触环境或者获知输入范围的情况下窃取深度强化学习策略，并提出了一种降低攻击效果的对策。

隐蔽仿真：奖励引导的环境无关策略窃取

Stealthy Imitation: Reward-guided Environment-free Policy Stealing

In the context of temporal image forensics, it is not evident that a neural
network, trained on images from different time-slots (classes), exploit solely
age related features. Usually, images taken in close temporal proximity (e.g.,
belonging to the same age class) share some common content properties. Such
content bias can be exploited by a neural network. In this work, a novel
approach that evaluates the influence of image content is proposed. This
approach is verified using synthetic images (where content bias can be ruled
out) with an age signal embedded. Based on the proposed approach, it is shown
that a `standard' neural network trained in the context of age classification
is strongly dependent on image content. As a potential countermeasure, two
different techniques are applied to mitigate the influence of the image content
during training, and they are also evaluated by the proposed method.

该研究探讨了时域图像取证中神经网络对图像内容的依赖性以及两种减轻该影响的技术的有效性。

深度学习年龄估计中的内容偏差：一种更具解释性的新方法

Content Bias in Deep Learning Age Approximation: A new Approach Towards  more Explainability

Poisoning attack is identified as a severe security threat to machine
learning algorithms. In many applications, for example, deep neural network
(DNN) models collect public data as the inputs to perform re-training, where
the input data can be poisoned. Although poisoning attack against support
vector machines (SVM) has been extensively studied before, there is still very
limited knowledge about how such attack can be implemented on neural networks
(NN), especially DNNs. In this work, we first examine the possibility of
applying traditional gradient-based method (named as the direct gradient
method) to generate poisoned data against NNs by leveraging the gradient of the
target model w.r.t. the normal data. We then propose a generative method to
accelerate the generation rate of the poisoned data: an auto-encoder
(generator) used to generate poisoned data is updated by a reward function of
the loss, and the target NN model (discriminator) receives the poisoned data to
calculate the loss w.r.t. the normal data. Our experiment results show that the
generative method can speed up the poisoned data generation rate by up to
239.38x compared with the direct gradient method, with slightly lower model
accuracy degradation. A countermeasure is also designed to detect such
poisoning attack methods by checking the loss of the target model.

本论文研究了在机器学习算法中，特别是深度神经网络中毒攻击的方法，提出了生成毒瘤数据的生成方法，并设计了一种检测方法来检测这种攻击。实验结果表明，与直接梯度法相比，这种方法可以加速毒瘤数据的生成速度高达 239.38 倍，且模型的准确度下降略微较低。