The growth and success of deep learning approaches can be attributed to two major factors: availability of hardware resources and availability of large number of training samples. For problems with large training databases, deep learning models have achieved superlative performances. However, there are a lot of \textit{small sample size or $S^3$} problems for which it is not feasible to collect large training databases. It has been observed that deep learning models do not generalize well on $S^3$ problems and specialized solutions are required. In this paper, we first present a review of deep learning algorithms for small sample size problems in which the algorithms are segregated according to the space in which they operate, i.e. input space, model space, and feature space. Secondly, we present Dynamic Attention Pooling approach which focuses on extracting global information from the most discriminative sub-part of the feature map. The performance of the proposed dynamic attention pooling is analyzed with state-of-the-art ResNet model on relatively small publicly available datasets such as SVHN, C10, C100, and TinyImageNet.

本文从输入空间、模型空间和特征空间三个角度出发，综述深度学习中针对小样本问题的算法，特别地介绍了动态注意力池化方法。该方法通过对特征图最具差异性的子区域进行全局信息的提取，显著提升了ResNet模型在SVHN、C10、C100和TinyImageNet这些小规模公开数据集上的性能。

深度学习中小样本量问题的探究