An underlying mechanism for successful deep learning (DL) with a limited deep
architecture and dataset, namely VGG-16 on CIFAR-10, was recently presented
based on a quantitative method to measure the quality of a single filter in
each layer. In this method, each filter identifies small clusters of possible
output labels, with additional noise selected as labels out of the clusters.
This feature is progressively sharpened with the layers, resulting in an
enhanced signal-to-noise ratio (SNR) and higher accuracy. In this study, the
suggested universal mechanism is verified for VGG-16 and EfficientNet-B0
trained on the CIFAR-100 and ImageNet datasets with the following main results.
First, the accuracy progressively increases with the layers, whereas the noise
per filter typically progressively decreases. Second, for a given deep
architecture, the maximal error rate increases approximately linearly with the
number of output labels. Third, the average filter cluster size and the number
of clusters per filter at the last convolutional layer adjacent to the output
layer are almost independent of the number of dataset labels in the range [3,
1,000], while a high SNR is preserved. The presented DL mechanism suggests
several techniques, such as applying filter's cluster connections (AFCC), to
improve the computational complexity and accuracy of deep architectures and
furthermore pinpoints the simplification of pre-existing structures while
maintaining their accuracies.

通过量化测量每个层的单个滤波器的质量，基于 CIFAR-10 上的 VGG-16 的有限深度架构和数据集的成功深度学习的潜在机制被提出，并验证了对 CIFAR-100 和 ImageNet 数据集上训练的 VGG-16 和 EfficientNet-B0 的普遍机制，结果表明准确率随着层次的增加而逐渐提高，噪声逐渐减少，滤波器最大误差率与输出标签数量近似线性增加，滤波器簇的平均尺寸和每个滤波器上的簇数在 [3, 1,000] 范围内，几乎与数据集标签数无关，同时保持较高的信噪比。该研究为改善深度结构的计算复杂性和准确性提供了几种技术，如应用滤波器的簇连接（AFCC），并强调了在保持准确性的同时简化现有结构。