We propose a novel method to estimate a driver's points-of-gaze using a pair
of ordinary cameras mounted on the windshield and dashboard of a car. This is a
challenging problem due to the dynamics of traffic environments with 3D scenes
of unknown depths. This problem is further complicated by the volatile distance
between the driver and the camera system. To tackle these challenges, we
develop a novel convolutional network that simultaneously analyzes the image of
the scene and the image of the driver's face. This network has a camera
calibration module that can compute an embedding vector that represents the
spatial configuration between the driver and the camera system. This
calibration module improves the overall network's performance, which can be
jointly trained end to end.
We also address the lack of annotated data for training and evaluation by
introducing a large-scale driving dataset with point-of-gaze annotations. This
is an in situ dataset of real driving sessions in an urban city, containing
synchronized images of the driving scene as well as the face and gaze of the
driver. Experiments on this dataset show that the proposed method outperforms
various baseline methods, having the mean prediction error of 29.69 pixels,
which is relatively small compared to the $1280{\times}720$ resolution of the
scene camera.

我们提出了一种使用安装在车辆挡风玻璃和仪表盘上的普通摄像头来估计驾驶员凝视点的新方法。该方法通过开发一种卷积网络来同时分析场景图像和驾驶员面部图像，其中包含一个相机校准模块，可以计算表示驾驶员和摄像头系统之间空间配置的嵌入向量，该模块改善了网络的性能并可以进行端到端的联合训练。我们还引入了一种大规模的驾驶数据集，其中包含了真实驾驶情景的图像以及驾驶员面部和凝视数据，对该数据集的实验表明，所提出的方法优于各种基准方法，均方误差为 29.69 像素，相对于场景摄像头的 $1280 {imes} 720$ 分辨率而言，误差较小。