Several recent works on self-supervised learning are trained by mapping different augmentations of the same image to the same feature representation. The set of used data augmentations is of crucial importance for the quality of the learned feature representation. We analyze how the traditionally used color jitter negatively impacts the quality of the color features in the learned feature representation. To address this problem, we replace this module with physics-based color augmentation, called Planckian jitter, which creates realistic variations in chromaticity, producing a model robust to llumination changes that can be commonly observed in real life, while maintaining the ability to discriminate the image content based on color information. We further improve the performance by introducing a latent space combination of color-sensitive and non-color-sensitive features. These are found to be complementary and the combination leads to large absolute performance gains over the default data augmentation on color classification tasks, including on Flowers-102 (+15%), Cub200 (+11%), VegFru (+15%), and T1K+ (+12%). Finally, we present a color sensitivity analysis to document the impact of different training methods on the model neurons and we show that the performance of the learned features is robust with respect to illuminant variations.

本文研究了自监督学习中数据增强的重要性。作者提出一种基于物理模型的 Planckian Jitter 数据增强方法，改善了传统颜色Jitter方法带来的颜色特征表达质量下降的问题，提高了学习到的特征表达的性能，尤其是在颜色信息方面。该增强方法在实际应用中可以解决光照变化对图像识别的影响，并且该方法的特征表达对不同的光照变化具有较强的不变性，具有很好的应用前景。

普朗克抖动：对自监督训练中颜色抖动的颜色削弱效应进行对抗