Machine learning is becoming increasingly popular as a go-to approach for
many tasks due to its world-class results. As a result, antivirus developers
are incorporating machine learning models into their products. While these
models improve malware detection capabilities, they also carry the disadvantage
of being susceptible to adversarial attacks. Although this vulnerability has
been demonstrated for many models in white-box settings, a black-box attack is
more applicable in practice for the domain of malware detection. We present a
generator of adversarial malware examples using reinforcement learning
algorithms. The reinforcement learning agents utilize a set of
functionality-preserving modifications, thus creating valid adversarial
examples. Using the proximal policy optimization (PPO) algorithm, we achieved
an evasion rate of 53.84% against the gradient-boosted decision tree (GBDT)
model. The PPO agent previously trained against the GBDT classifier scored an
evasion rate of 11.41% against the neural network-based classifier MalConv and
an average evasion rate of 2.31% against top antivirus programs. Furthermore,
we discovered that random application of our functionality-preserving portable
executable modifications successfully evades leading antivirus engines, with an
average evasion rate of 11.65%. These findings indicate that machine
learning-based models used in malware detection systems are vulnerable to
adversarial attacks and that better safeguards need to be taken to protect
these systems.

应用强化学习算法生成对抗性恶意文件可以有效绕过大多数杀毒软件，揭示了基于机器学习的恶意软件检测系统易受对抗性攻击的脆弱性。