Input-Convex Neural Networks (ICNNs) are networks that guarantee convexity in their input-output mapping. These networks have been successfully applied for energy-based modelling, optimal transport problems and learning invariances. The convexity of ICNNs is achieved by using non-decreasing convex activation functions and non-negative weights. Because of these peculiarities, previous initialisation strategies, which implicitly assume centred weights, are not effective for ICNNs. By studying signal propagation through layers with non-negative weights, we are able to derive a principled weight initialisation for ICNNs. Concretely, we generalise signal propagation theory by removing the assumption that weights are sampled from a centred distribution. In a set of experiments, we demonstrate that our principled initialisation effectively accelerates learning in ICNNs and leads to better generalisation. Moreover, we find that, in contrast to common belief, ICNNs can be trained without skip-connections when initialised correctly. Finally, we apply ICNNs to a real-world drug discovery task and show that they allow for more effective molecular latent space exploration.

输入凸性神经网络（ICNN）是能够保证输入-输出映射凸性的网络。本文通过对非负权重层的信号传播研究，推导出一种基于原理的ICNN权重初始化方法，实验证明这种初始方法有效加速了ICNN的学习和提高了泛化性能。此外，ICNN还被应用于药物发现任务，可以更有效地进行分子潜在空间的探索。

输入凸神经网络的基本权重初始化