Visual Saliency refers to the innate human mechanism of focusing on and
extracting important features from the observed environment. Recently, there
has been a notable surge of interest in the field of automotive research
regarding the estimation of visual saliency. While operating a vehicle, drivers
naturally direct their attention towards specific objects, employing
brain-driven saliency mechanisms that prioritize certain elements over others.
In this investigation, we present an intelligent system that combines a
drowsiness detection system for drivers with a scene comprehension pipeline
based on saliency. To achieve this, we have implemented a specialized 3D deep
network for semantic segmentation, which has been pretrained and tailored for
processing the frames captured by an automotive-grade external camera. The
proposed pipeline was hosted on an embedded platform utilizing the STA1295
core, featuring ARM A7 dual-cores, and embeds an hardware accelerator.
Additionally, we employ an innovative biosensor embedded on the car steering
wheel to monitor the driver drowsiness, gathering the PhotoPlethysmoGraphy
(PPG) signal of the driver. A dedicated 1D temporal deep convolutional network
has been devised to classify the collected PPG time-series, enabling us to
assess the driver level of attentiveness. Ultimately, we compare the determined
attention level of the driver with the corresponding saliency-based scene
classification to evaluate the overall safety level. The efficacy of the
proposed pipeline has been validated through extensive experimental results.

我们提出了一种结合司机困倦检测系统和基于显著性的场景理解流水线的智能系统，该系统在嵌入式平台上运行，并利用专用 3D 深度网络进行语义分割。我们还使用车辆方向盘上嵌入的创新生物传感器来监测驾驶员的困倦程度，并对收集到的驾驶员脉搏波信号进行分类，从而评估驾驶员的注意力水平。最后，我们将司机的注意力水平与基于显著性的场景分类进行比较，以评估整体安全水平。通过广泛的实验结果验证了该流水线的有效性。