The emerging task of fine-grained image classification in low-data regimes
assumes the presence of low inter-class variance and large intra-class
variation along with a highly limited amount of training samples per class.
However, traditional ways of separately dealing with fine-grained
categorisation and extremely scarce data may be inefficient under both these
harsh conditions presented together. In this paper, we present a novel
framework, called AD-Net, aiming to enhance deep neural network performance on
this challenge by leveraging the power of Augmentation and Distillation
techniques. Specifically, our approach is designed to refine learned features
through self-distillation on augmented samples, mitigating harmful overfitting.
We conduct comprehensive experiments on popular fine-grained image
classification benchmarks where our AD-Net demonstrates consistent improvement
over traditional fine-tuning and state-of-the-art low-data techniques.
Remarkably, with the smallest data available, our framework shows an
outstanding relative accuracy increase of up to 45 % compared to standard
ResNet-50 and up to 27 % compared to the closest SOTA runner-up. We emphasise
that our approach is practically architecture-independent and adds zero extra
cost at inference time. Additionally, we provide an extensive study on the
impact of every framework's component, highlighting the importance of each in
achieving optimal performance. Source code and trained models are publicly
available at github.com/demidovd98/fgic_lowd.

通过利用增强和蒸馏技术，AD-Net 框架通过自我蒸馏技术在增强样本上优化学习特征，减轻有害的过拟合，从而在弱数据条件下提高深度神经网络性能，相对于标准 ResNet-50 和最先进的低数据技术的 SOTA 接近者，我们的框架相对准确度最高可达到 45％和 27％，我们强调我们的方法在体系结构上是独立的，并且在推理时间上不增加任何额外成本。