This work introduces a new multispectral database and novel approaches for eyeblink detection in RGB and Near-Infrared (NIR) individual images. Our contributed dataset (mEBAL2, multimodal Eye Blink and Attention Level estimation, Version 2) is the largest existing eyeblink database, representing a great opportunity to improve data-driven multispectral approaches for blink detection and related applications (e.g., attention level estimation and presentation attack detection in face biometrics). mEBAL2 includes 21,100 image sequences from 180 different students (more than 2 million labeled images in total) while conducting a number of e-learning tasks of varying difficulty or taking a real course on HTML initiation through the edX MOOC platform. mEBAL2 uses multiple sensors, including two Near-Infrared (NIR) and one RGB camera to capture facial gestures during the execution of the tasks, as well as an Electroencephalogram (EEG) band to get the cognitive activity of the user and blinking events. Furthermore, this work proposes a Convolutional Neural Network architecture as benchmark for blink detection on mEBAL2 with performances up to 97%. Different training methodologies are implemented using the RGB spectrum, NIR spectrum, and the combination of both to enhance the performance on existing eyeblink detectors. We demonstrate that combining NIR and RGB images during training improves the performance of RGB eyeblink detectors (i.e., detection based only on a RGB image). Finally, the generalization capacity of the proposed eyeblink detectors is validated in wilder and more challenging environments like the HUST-LEBW dataset to show the usefulness of mEBAL2 to train a new generation of data-driven approaches for eyeblink detection.

这项研究介绍了一个新的多光谱数据库和针对 RGB 和近红外（NIR）个体图像的眨眼检测的新方法。通过提供的数据集（mEBAL2），我们有机会改进数据驱动的多光谱方法，以提高眨眼检测和相关应用的表现。通过在各种难度的在线学习任务中使用多个传感器捕捉面部动作的方式，以及结合脑电图（EEG）频段记录用户的认知活动和眨眼事件，我们证明了这些方法的泛化能力。最终，通过在更具挑战性的环境中验证模型的适用性，我们展示了训练mEBAL2可以用于培养新一代数据驱动的眨眼检测方法的重要性。

mEBAL2数据库与基准：基于图像的多光谱眨眼检测