There have been many new methods and tricks to make creating an architecture, more robust and perform better in the last several years. While these methods are proved to indeed contribute to better performance, they impose some restrictions as well, such as memory consumption that makes their application in deeper networks not always possible[1]. Having this in mind, in this work, we propose a simple architecture for convolutional neural network that not only achieves better performance and accuracy and imposes less computational cost than the latest well-known deeper architectures such as VGGNet, ResNet, in the several recognition benchmark datasets but can be trained more easily as well. We also introduce a new trick we call leap-out which along with the new architecture, let us achieve state of the art and very close to the state of the art it in benchmarking datasets such as Cifar10, cifar100 and mnist.

提出了一种简单的卷积神经网络架构(SimpleNet)，在保持计算和内存效率的同时，具有与更复杂的CNN相媲美的性能，在多个基准测试中表现出色，包括CIFAR-10，MNIST，CIFAR-100和SVHN，甚至在ImageNet数据集中也优于许多更大更深的CNN模型。

简单为上，简单架构超越深度和复杂架构