While fine-tuning is a de facto standard method for training deep neural
networks, it still suffers from overfitting when using small target datasets.
Previous methods improve fine-tuning performance by maintaining knowledge of
the source datasets or introducing regularization terms such as contrastive
loss. However, these methods require auxiliary source information (e.g., source
labels or datasets) or heavy additional computations. In this paper, we propose
a simple method called adaptive random feature regularization (AdaRand).
AdaRand helps the feature extractors of training models to adaptively change
the distribution of feature vectors for downstream classification tasks without
auxiliary source information and with reasonable computation costs. To this
end, AdaRand minimizes the gap between feature vectors and random reference
vectors that are sampled from class conditional Gaussian distributions.
Furthermore, AdaRand dynamically updates the conditional distribution to follow
the currently updated feature extractors and balance the distance between
classes in feature spaces. Our experiments show that AdaRand outperforms the
other fine-tuning regularization, which requires auxiliary source information
and heavy computation costs.

使用自适应随机特征正则化 (AdaRand) 方法，无需辅助源信息和重大计算成本，通过改变特征向量的分布来提高训练模型的下游分类任务性能。