In the burgeoning field of intelligent transportation systems, enhancing
vehicle-driver interaction through facial attribute recognition, such as facial
expression, eye gaze, age, etc., is of paramount importance for safety,
personalization, and overall user experience. However, the scarcity of
comprehensive large-scale, real-world datasets poses a significant challenge
for training robust multi-task models. Existing literature often overlooks the
potential of synthetic datasets and the comparative efficacy of
state-of-the-art vision foundation models in such constrained settings. This
paper addresses these gaps by investigating the utility of synthetic datasets
for training complex multi-task models that recognize facial attributes of
passengers of a vehicle, such as gaze plane, age, and facial expression.
Utilizing transfer learning techniques with both pre-trained Vision Transformer
(ViT) and Residual Network (ResNet) models, we explore various training and
adaptation methods to optimize performance, particularly when data availability
is limited. We provide extensive post-evaluation analysis, investigating the
effects of synthetic data distributions on model performance in in-distribution
data and out-of-distribution inference. Our study unveils counter-intuitive
findings, notably the superior performance of ResNet over ViTs in our specific
multi-task context, which is attributed to the mismatch in model complexity
relative to task complexity. Our results highlight the challenges and
opportunities for enhancing the use of synthetic data and vision foundation
models in practical applications.

通过研究合成数据集在训练复杂的多任务模型中的实用性，特别是在数据可用性有限的情况下，以及视觉基础模型的比较功效，本研究揭示了关于合成数据和视觉基础模型在实际应用中的挑战和机遇。

面向车载多任务人脸属性识别的研究：探究合成数据和视觉基础模型

Towards In-Vehicle Multi-Task Facial Attribute Recognition:  Investigating Synthetic Data and Vision Foundation Models

Recognising detailed facial or clothing attributes in images of people is a
challenging task for computer vision, especially when the training data are
both in very large scale and extremely imbalanced among different attribute
classes. To address this problem, we formulate a novel scheme for batch
incremental hard sample mining of minority attribute classes from imbalanced
large scale training data. We develop an end-to-end deep learning framework
capable of avoiding the dominant effect of majority classes by discovering
sparsely sampled boundaries of minority classes. This is made possible by
introducing a Class Rectification Loss (CRL) regularising algorithm. We
demonstrate the advantages and scalability of CRL over existing
state-of-the-art attribute recognition and imbalanced data learning models on
two large scale imbalanced benchmark datasets, the CelebA facial attribute
dataset and the X-Domain clothing attribute dataset.

通过引入 Class Rectification Loss 正则化算法，利用 batch incremental hard sample mining 方案，在大规模训练数据中从少数属性类中挖掘极端样本，并发展出一种避免多数类占主导地位的端到端深度学习框架，可高效识别人脸或衣物等细节属性。试验结果显示，该框架在两个大规模标准数据集 CelebA 和 X-Domain 上，相对于现有的属性识别和不平衡数据学习模型，具有更好的优势和可扩展性。

基于矩形校正的困难样本挖掘方法用于不平衡深度学习

Class Rectification Hard Mining for Imbalanced Deep Learning

Attribute recognition, particularly facial, extracts many labels for each
image. While some multi-task vision problems can be decomposed into separate
tasks and stages, e.g., training independent models for each task, for a
growing set of problems joint optimization across all tasks has been shown to
improve performance. We show that for deep convolutional neural network (DCNN)
facial attribute extraction, multi-task optimization is better. Unfortunately,
it can be difficult to apply joint optimization to DCNNs when training data is
imbalanced, and re-balancing multi-label data directly is structurally
infeasible, since adding/removing data to balance one label will change the
sampling of the other labels. This paper addresses the multi-label imbalance
problem by introducing a novel mixed objective optimization network (MOON) with
a loss function that mixes multiple task objectives with domain adaptive
re-weighting of propagated loss. Experiments demonstrate that not only does
MOON advance the state of the art in facial attribute recognition, but it also
outperforms independently trained DCNNs using the same data. When using facial
attributes for the LFW face recognition task, we show that our balanced (domain
adapted) network outperforms the unbalanced trained network.

本文提出了一种新的混合目标优化网络，其中的损失函数混合了多种任务目标，并具有领域自适应的传播损失重新加权，解决了深度卷积神经网络中的多标签不平衡问题，该方法在面部属性识别方面表现良好，优于使用相同数据的独立训练的 DCNN。