Previous work in Neural Loss Function Search (NLFS) has shown a lack of
correlation between smaller surrogate functions and large convolutional neural
networks with massive regularization. We expand upon this research by revealing
another disparity that exists, correlation between different types of image
augmentation techniques. We show that different loss functions can perform well
on certain image augmentation techniques, while performing poorly on others. We
exploit this disparity by performing an evolutionary search on five types of
image augmentation techniques in the hopes of finding image augmentation
specific loss functions. The best loss functions from each evolution were then
taken and transferred to WideResNet-28-10 on CIFAR-10 and CIFAR-100 across each
of the five image augmentation techniques. The best from that were then taken
and evaluated by fine-tuning EfficientNetV2Small on the CARS, Oxford-Flowers,
and Caltech datasets across each of the five image augmentation techniques.
Multiple loss functions were found that outperformed cross-entropy across
multiple experiments. In the end, we found a single loss function, which we
called the inverse bessel logarithm loss, that was able to outperform
cross-entropy across the majority of experiments.

通过进行演化搜索找到了一种名为逆贝塞尔对数损失的单一损失函数，它在多数实验中表现优于交叉熵。

针对特定图像增强技术演化的损失函数

Evolving Loss Functions for Specific Image Augmentation Techniques

For classification, neural networks typically learn by minimizing
cross-entropy, but are evaluated and compared using accuracy. This disparity
suggests neural loss function search (NLFS), the search for a drop-in
replacement loss function of cross-entropy for neural networks. We apply NLFS
to image classifier convolutional neural networks. We propose a new search
space for NLFS that encourages more diverse loss functions to be explored, and
a surrogate function that accurately transfers to large-scale convolutional
neural networks. We search the space using regularized evolution, a
mutation-only aging genetic algorithm. After evolution and a proposed loss
function elimination protocol, we transferred the final loss functions across
multiple architectures, datasets, and image augmentation techniques to assess
generalization. In the end, we discovered three new loss functions, called
NeuroLoss1, NeuroLoss2, and NeuroLoss3 that were able to outperform
cross-entropy in terms of a higher mean test accuracy as a simple drop-in
replacement loss function across the majority of experiments.

使用神经损失函数搜索（NLFS）方法，我们在卷积神经网络中发现了三个新的损失函数（NeuroLoss1、NeuroLoss2 和 NeuroLoss3），它们能够以更高的平均测试准确率取代交叉熵作为简单的损失函数，从而实现更好的泛化效果。