We propose a novel way to incorporate expert knowledge into the training of deep neural networks. Many approaches encode domain constraints directly into the network architecture, requiring non-trivial or domain-specific engineering. In contrast, our approach, called MultiplexNet, represents domain knowledge as a logical formula in disjunctive normal form (DNF) which is easy to encode and to elicit from human experts. It introduces a Categorical latent variable that learns to choose which constraint term optimizes the error function of the network and it compiles the constraints directly into the output of existing learning algorithms. We demonstrate the efficacy of this approach empirically on several classical deep learning tasks, such as density estimation and classification in both supervised and unsupervised settings where prior knowledge about the domains was expressed as logical constraints. Our results show that the MultiplexNet approach learned to approximate unknown distributions well, often requiring fewer data samples than the alternative approaches. In some cases, MultiplexNet finds better solutions than the baselines; or solutions that could not be achieved with the alternative approaches. Our contribution is in encoding domain knowledge in a way that facilitates inference that is shown to be both efficient and general; and critically, our approach guarantees 100% constraint satisfaction in a network's output.

我们提出了一种新颖的方法，将专家知识纳入深度神经网络的训练中，并将领域知识表示为易于编码和从人类专家那里引出的析取范式逻辑公式。该方法引入了一个分类潜变量来学习选择哪个约束项最优化网络的误差函数，并将约束条件直接编译到现有学习算法的输出中。我们在几个经典的深度学习任务中实证了该方法的功效，结果表明MultiplexNet方法能够良好地逼近未知的分布，通常需要比其他方法更少的数据样本，并且在某些情况下，具有比其他方法更好的解决方案。

MultiplexNet: 神经网络中完全满足逻辑约束的探索