Natural interaction with virtual objects in AR/VR environments makes for a
smooth user experience. Gestures are a natural extension from real world to
augmented space to achieve these interactions. Finding discriminating
spatio-temporal features relevant to gestures and hands in ego-view is the
primary challenge for recognising egocentric gestures. In this work we propose
a data driven end-to-end deep learning approach to address the problem of
egocentric gesture recognition, which combines an ego-hand encoder network to
find ego-hand features, and a recurrent neural network to discern temporally
discriminating features. Since deep learning networks are data intensive, we
propose a novel data augmentation technique using green screen capture to
alleviate the problem of ground truth annotation. In addition we publish a
dataset of 10 gestures performed in a natural fashion in front of a green
screen for training and the same 10 gestures performed in different natural
scenes without green screen for validation. We also present the results of our
network's performance in comparison to the state-of-the-art using the AirGest
dataset

本文提出一种基于端到端深度学习方法，采用用于识别土生土长的姿势的时间分辨率特征的循环神经网络和用于找到土手特征的自我手编码器网络，结合绿色屏幕捕获的数据扩充技术来缓解地面真实性注释问题，从而实现了土心姿势识别的问题。