For driver observation frameworks, clean datasets collected in controlled
simulated environments often serve as the initial training ground. Yet, when
deployed under real driving conditions, such simulator-trained models quickly
face the problem of distributional shifts brought about by changing
illumination, car model, variations in subject appearances, sensor
discrepancies, and other environmental alterations.
This paper investigates the viability of transferring video-based driver
observation models from simulation to real-world scenarios in autonomous
vehicles, given the frequent use of simulation data in this domain due to
safety issues. To achieve this, we record a dataset featuring actual autonomous
driving conditions and involving seven participants engaged in highly
distracting secondary activities. To enable direct SIM to REAL transfer, our
dataset was designed in accordance with an existing large-scale simulator
dataset used as the training source. We utilize the Inflated 3D ConvNet (I3D)
model, a popular choice for driver observation, with Gradient-weighted Class
Activation Mapping (Grad-CAM) for detailed analysis of model decision-making.
Though the simulator-based model clearly surpasses the random baseline, its
recognition quality diminishes, with average accuracy dropping from 85.7% to
46.6%. We also observe strong variations across different behavior classes.
This underscores the challenges of model transferability, facilitating our
research of more robust driver observation systems capable of dealing with real
driving conditions.

研究通过模拟数据进行驾驶员观察模型的训练，并探索将这些模型转移到真实自动驾驶场景中的可行性，结果显示模型的识别质量明显下降，存在模型转移的挑战。