In the research community of continuous hand gesture recognition (HGR), the current publicly available datasets lack real-world elements needed to build responsive and efficient HGR systems. In this paper, we introduce a new benchmark dataset named IPN Hand with sufficient size, variation, and real-world elements able to train and evaluate deep neural networks. This dataset contains more than 4 000 gesture samples and 800 000 RGB frames from 50 distinct subjects. We design 13 different static and dynamic gestures focused on interaction with touchless screens. We especially consider the scenario when continuous gestures are performed without transition states, and when subjects perform natural movements with their hands as non-gesture actions. Gestures were collected from about 30 diverse scenes, with real-world variation in background and illumination. With our dataset, the performance of three 3D-CNN models is evaluated on the tasks of isolated and continuous real-time HGR. Furthermore, we analyze the possibility of increasing the recognition accuracy by adding multiple modalities derived from RGB frames, i.e., optical flow and semantic segmentation, while keeping the real-time performance of the 3D-CNN model. Our empirical study also provides a comparison with the publicly available nvGesture (NVIDIA) dataset. The experimental results show that the state-of-the-art ResNext-101 model decreases about 30% accuracy when using our real-world dataset, demonstrating that the IPN Hand dataset can be used as a benchmark, and may help the community to step forward in the continuous HGR. Our dataset and pre-trained models used in the evaluation are publicly available at https://github.com/GibranBenitez/IPN-hand.

本文介绍了一个名为IPN Hand的新的基准数据集，用于训练和评估深度神经网络，包含超过4,000个姿势样本和50个人的800,000个RGB帧。作者对13个不同的静态和动态手势进行了设计以应对无触摸屏幕的交互，并考虑了手部自然动作。作者评估了三种3D-CNN模型在孤立和连续实时HGR任务上的性能，并探讨了增加识别准确性的可能性。作者的实验结果表明，最先进的ResNext-101模型在使用其真实世界数据集时准确率降低约30％，表明IPN Hand数据集可以用作基准，并有助于推动连续HGR的发展。

IPN Hand: 一个视频数据集和基准，用于实时连续手势识别