Deep neural networks is a branch in machine learning that has seen a meteoric
rise in popularity due to its powerful abilities to represent and model
high-level abstractions in highly complex data. One area in deep neural
networks that is ripe for exploration is neural connectivity formation. A
pivotal study on the brain tissue of rats found that synaptic formation for
specific functional connectivity in neocortical neural microcircuits can be
surprisingly well modeled and predicted as a random formation. Motivated by
this intriguing finding, we introduce the concept of StochasticNet, where deep
neural networks are formed via stochastic connectivity between neurons. As a
result, any type of deep neural networks can be formed as a StochasticNet by
allowing the neuron connectivity to be stochastic. Stochastic synaptic
formations, in a deep neural network architecture, can allow for efficient
utilization of neurons for performing specific tasks. To evaluate the
feasibility of such a deep neural network architecture, we train a
StochasticNet using four different image datasets (CIFAR-10, MNIST, SVHN, and
STL-10). Experimental results show that a StochasticNet, using less than half
the number of neural connections as a conventional deep neural network,
achieves comparable accuracy and reduces overfitting on the CIFAR-10, MNIST and
SVHN dataset. Interestingly, StochasticNet with less than half the number of
neural connections, achieved a higher accuracy (relative improvement in test
error rate of ~6% compared to ConvNet) on the STL-10 dataset than a
conventional deep neural network. Finally, StochasticNets have faster
operational speeds while achieving better or similar accuracy performances.

本研究旨在利用 StochasticNet 的随机连接性形成神经网络的结构，用以提高神经网络对特定任务的执行能力和减少过度拟合，并在 CIFAR-10，MNIST，SVHN 和 STL-10 四个数据集上训练 StochasticNet，结果表明 StochasticNet 在数据集上展现出了比传统神经网络更好或者相似的准确性能。