In this work, we introduce Y-Drop, a regularization method that biases the dropout algorithm towards dropping more important neurons with higher probability. The backbone of our approach is neuron conductance, an interpretable measure of neuron importance that calculates the contribution of each neuron towards the end-to-end mapping of the network. We investigate the impact of the uniform dropout selection criterion on performance by assigning higher dropout probability to the more important units. We show that forcing the network to solve the task at hand in the absence of its important units yields a strong regularization effect. Further analysis indicates that Y-Drop yields solutions where more neurons are important, i.e have high conductance, and yields robust networks. In our experiments we show that the regularization effect of Y-Drop scales better than vanilla dropout w.r.t. the architecture size and consistently yields superior performance over multiple datasets and architecture combinations, with little tuning.

本研究提出了一种新颖的正则化方法Y-Drop，旨在通过提高重要神经元的丢弃概率来克服传统dropout方法的不足。通过引入神经元导电性作为衡量重要性的可解释指标，我们展示了这种方法在多种架构和数据集上的优越性能，显著提升了网络的鲁棒性。

Y-Drop：一种基于导电性的全连接层 dropout 方法