We propose a method to impose homogeneous linear inequality constraints of
the form $Ax\leq 0$ on neural network activations. The proposed method allows a
data-driven training approach to be combined with modeling prior knowledge
about the task. One way to achieve this task is by means of a projection step
at test time after unconstrained training. However, this is an expensive
operation. By directly incorporating the constraints into the architecture, we
can significantly speed-up inference at test time; for instance, our
experiments show a speed-up of up to two orders of magnitude over a projection
method. Our algorithm computes a suitable parameterization of the feasible set
at initialization and uses standard variants of stochastic gradient descent to
find solutions to the constrained network. Thus, the modeling constraints are
always satisfied during training. Crucially, our approach avoids to solve an
optimization problem at each training step or to manually trade-off data and
constraint fidelity with additional hyperparameters. We consider constrained
generative modeling as an important application domain and experimentally
demonstrate the proposed method by constraining a variational autoencoder.

本文提出了一种方法来在神经网络激活中施加均匀的线性不等式约束，以结合数据驱动的训练方法和先前任务的先验知识。该算法可在测试时间内显著加速推理速度，并避免了在每个训练步骤中求解优化问题或手动调整超参数。该方法通过约束变分自编码器进行实验演示。