Neural networks trained on standard image classification data sets are shown to be less resistant to data set bias. It is necessary to comprehend the behavior objective function that might correspond to superior performance for data with biases. However, there is little research on the selection of the objective function and its representational structure when trained on data set with biases. In this paper, we investigate the performance and internal representational structure of convolution-based neural networks (e.g., ResNets) trained on biased data using various objective functions. We specifically study similarities in representations, using Centered Kernel Alignment (CKA), for different objective functions (probabilistic and margin-based) and offer a comprehensive analysis of the chosen ones. According to our findings, ResNets representations obtained with Negative Log Likelihood $(\mathcal{L}_{NLL})$ and Softmax Cross-Entropy ($\mathcal{L}_{SCE}$) as loss functions are equally capable of producing better performance and fine representations on biased data. We note that without progressive representational similarities among the layers of a neural network, the performance is less likely to be robust.

研究了使用不同目标函数，基于卷积神经网络（如ResNets）训练偏重数据集的性能和内部表征结构，表明负对数似然（NLL）和softmax交叉熵（SCE）作为损失函数得到的ResNets表征同样能够在偏重数据上产生更好的性能和细致替代，并且在神经网络的各层之间具有渐进表征相似性，性能更有可能是稳健的。

神经网络中的偏见解释：探究表征相似性