Deep Neural Networks (DNNs) are very popular these days, and are the subject of a very intense investigation. A DNN is made by layers of internal units (or neurons), each of which computes an affine combination of the output of the units in the previous layer, applies a nonlinear operator, and outputs the corresponding value (also known as activation). A commonly-used nonlinear operator is the so-called rectified linear unit (ReLU), whose output is just the maximum between its input value and zero. In this (and other similar cases like max pooling, where the max operation involves more than one input value), one can model the DNN as a 0-1 Mixed Integer Linear Program (0-1 MILP) where the continuous variables correspond to the output values of each unit, and a binary variable is associated with each ReLU to model its yes/no nature. In this paper we discuss the peculiarity of this kind of 0-1 MILP models, and describe an effective bound-tightening technique intended to ease its solution. We also present possible applications of the 0-1 MILP model arising in feature visualization and in the construction of adversarial examples. Preliminary computational results are reported, aimed at investigating (on small DNNs) the computational performance of a state-of-the-art MILP solver when applied to a known test case, namely, hand-written digit recognition.

本文旨在探讨深度神经网络以及与之相关的0-1混合整数线性规划模型，并描述了一种有效的边界紧缩技术，旨在促进其解决方案。文中还介绍了0-1混合整数线性规划模型在特征可视化和对抗样本构建中的可能应用，并针对手写数字识别这一已知的测试用例，报告了关于使用最先进的混合整数线性规划求解器在小规模DNN上的计算性能的初步结果。

深度神经网络作为0-1混合整数线性规划问题：可行性研究