Stochastic regularization of neural networks (e.g. dropout) is a wide-spread technique in deep learning that allows for better generalization. Despite its success, continuous-time models, such as neural ordinary differential equation (ODE), usually rely on a completely deterministic feed-forward operation. This work provides an empirical study of stochastically regularized neural ODE on several image-classification tasks (CIFAR-10, CIFAR-100, TinyImageNet). Building upon the formalism of stochastic differential equations (SDEs), we demonstrate that neural SDE is able to outperform its deterministic counterpart. Further, we show that data augmentation during the training improves the performance of both deterministic and stochastic versions of the same model. However, the improvements obtained by the data augmentation completely eliminate the empirical gains of the stochastic regularization, making the difference in the performance of neural ODE and neural SDE negligible.

本文对几个图像分类任务进行了随机正则化神经 ODE 的实证研究，探讨了数据增强对其性能的影响，展示了神经 SDE 相对于其确定性版本的优势，但进一步的研究表明，数据增强消除了随机正则化的影响，使得神经ODE和神经SDE的性能差异微不足道。

神经常微分方程中的随机性：实证研究