As machine-learning (ML) based systems for malware detection become more
prevalent, it becomes necessary to quantify the benefits compared to the more
traditional anti-virus (AV) systems widely used today. It is not practical to
build an agreed upon test set to benchmark malware detection systems on pure
classification performance. Instead we tackle the problem by creating a new
testing methodology, where we evaluate the change in performance on a set of
known benign & malicious files as adversarial modifications are performed. The
change in performance combined with the evasion techniques then quantifies a
system's robustness against that approach. Through these experiments we are
able to show in a quantifiable way how purely ML based systems can be more
robust than AV products at detecting malware that attempts evasion through
modification, but may be slower to adapt in the face of significantly novel
attacks.

通过创建新的测试方法，我们评估了针对一组已知的 Benign & Malicious 文件执行敌对修改并评估性能变化的变化以及逃避技术，从而依靠实验证明了仅基于 ML 的系统可以比 AV 产品更有效地检测到试图通过修改来逃避的恶意软件，但在面临显着的新型攻击时可能反应较慢。