Machine learning models are notoriously difficult to interpret and debug.
This is particularly true of neural networks. In this work, we introduce
automated software testing techniques for neural networks that are well-suited
to discovering errors which occur only for rare inputs. Specifically, we
develop coverage-guided fuzzing (CGF) methods for neural networks. In CGF,
random mutations of inputs to a neural network are guided by a coverage metric
toward the goal of satisfying user-specified constraints. We describe how fast
approximate nearest neighbor algorithms can provide this coverage metric. We
then discuss the application of CGF to the following goals: finding numerical
errors in trained neural networks, generating disagreements between neural
networks and quantized versions of those networks, and surfacing undesirable
behavior in character level language models. Finally, we release an open source
library called TensorFuzz that implements the described techniques.

本研究介绍了用于发现神经网络中仅在罕见输入下出现错误的自动化软件测试技术，其中关键技术为基于覆盖率的模糊测试方法，并提出快速近似最近邻居算法来提供覆盖度量，最终开发了 TensorFuzz 开源库以实现这些技术。