Driver activity classification is crucial for ensuring road safety, with
applications ranging from driver assistance systems to autonomous vehicle
control transitions. In this paper, we present a novel approach leveraging
generalizable representations from vision-language models for driver activity
classification. Our method employs a Semantic Representation Late Fusion Neural
Network (SRLF-Net) to process synchronized video frames from multiple
perspectives. Each frame is encoded using a pretrained vision-language encoder,
and the resulting embeddings are fused to generate class probability
predictions. By leveraging contrastively-learned vision-language
representations, our approach achieves robust performance across diverse driver
activities. We evaluate our method on the Naturalistic Driving Action
Recognition Dataset, demonstrating strong accuracy across many classes. Our
results suggest that vision-language representations offer a promising avenue
for driver monitoring systems, providing both accuracy and interpretability
through natural language descriptors.

使用视觉 - 语言模型的通用表征，在司机活动分类中利用语义表示延迟融合神经网络（SRLF-Net）的新方法，通过对多个视角的同步视频帧进行处理，对每个帧进行预训练的视觉 - 语言编码器进行编码，并融合生成类别概率预测，利用对比学习的视觉 - 语言表示，方法在自然驾驶行为识别数据集上得到了鲁棒的性能，证明视觉 - 语言表示为司机监控系统提供了准确性和可解释性。