Neural networks trained on standard image classification data sets are shown
to be less resistant to data set bias. It is necessary to comprehend the
behavior objective function that might correspond to superior performance for
data with biases. However, there is little research on the selection of the
objective function and its representational structure when trained on data set
with biases.
In this paper, we investigate the performance and internal representational
structure of convolution-based neural networks (e.g., ResNets) trained on
biased data using various objective functions. We specifically study
similarities in representations, using Centered Kernel Alignment (CKA), for
different objective functions (probabilistic and margin-based) and offer a
comprehensive analysis of the chosen ones.
According to our findings, ResNets representations obtained with Negative Log
Likelihood $(\mathcal{L}_{NLL})$ and Softmax Cross-Entropy
($\mathcal{L}_{SCE}$) as loss functions are equally capable of producing better
performance and fine representations on biased data. We note that without
progressive representational similarities among the layers of a neural network,
the performance is less likely to be robust.

研究了使用不同目标函数，基于卷积神经网络（如 ResNets）训练偏重数据集的性能和内部表征结构，表明负对数似然（NLL）和 softmax 交叉熵（SCE）作为损失函数得到的 ResNets 表征同样能够在偏重数据上产生更好的性能和细致替代，并且在神经网络的各层之间具有渐进表征相似性，性能更有可能是稳健的。