The use of Mixed-Integer Linear Programming (MILP) models to represent neural
networks with Rectified Linear Unit (ReLU) activations has become increasingly
widespread in the last decade. This has enabled the use of MILP technology to
test-or stress-their behavior, to adversarially improve their training, and to
embed them in optimization models leveraging their predictive power. Many of
these MILP models rely on activation bounds. That is, bounds on the input
values of each neuron. In this work, we explore the tradeoff between the
tightness of these bounds and the computational effort of solving the resulting
MILP models. We provide guidelines for implementing these models based on the
impact of network structure, regularization, and rounding.

利用混合整数线性规划（MILP）模型来表示带有修正线性单元激活函数的神经网络的使用已在过去十年中越来越普遍。本研究探讨了这些边界的紧密性与求解结果 MILP 模型的计算付出之间的权衡，并提供了实施这些模型的指南，基于网络结构、正则化和四舍五入的影响。

基于优化紧束的 ReLU 网络的计算权衡

Computational Tradeoffs of Optimization-Based Bound Tightening in ReLU  Networks

Deep neural networks are vulnerable to backdoor attacks (Trojans), where an
attacker poisons the training set with backdoor triggers so that the neural
network learns to classify test-time triggers to the attacker's designated
target class. Recent work shows that backdoor poisoning induces over-fitting
(abnormally large activations) in the attacked model, which motivates a
general, post-training clipping method for backdoor mitigation, i.e., with
bounds on internal-layer activations learned using a small set of clean
samples. We devise a new such approach, choosing the activation bounds to
explicitly limit classification margins. This method gives superior performance
against peer methods for CIFAR-10 image classification. We also show that this
method has strong robustness against adaptive attacks, X2X attacks, and on
different datasets. Finally, we demonstrate a method extension for test-time
detection and correction based on the output differences between the original
and activation-bounded networks. The code of our method is online available.

深度神经网络容易受到后门攻击，通过限制内部激活层的界限，可以有效减轻此类攻击及提高分类性能，在测试时还可以检测和校正激活界限网络与原网络之间的输出差异。