Transformer model have achieved state-of-the-art results in many applications
like NLP, classification, etc. But their exploration in gesture recognition
task is still limited. So, we propose a novel GestFormer architecture for
dynamic hand gesture recognition. The motivation behind this design is to
propose a resource efficient transformer model, since transformers are
computationally expensive and very complex. So, we propose to use a pooling
based token mixer named PoolFormer, since it uses only pooling layer which is a
non-parametric layer instead of quadratic attention. The proposed model also
leverages the space-invariant features of the wavelet transform and also the
multiscale features are selected using multi-scale pooling. Further, a gated
mechanism helps to focus on fine details of the gesture with the contextual
information. This enhances the performance of the proposed model compared to
the traditional transformer with fewer parameters, when evaluated on dynamic
hand gesture datasets, NVidia Dynamic Hand Gesture and Briareo datasets. To
prove the efficacy of the proposed model, we have experimented on single as
well multimodal inputs such as infrared, normals, depth, optical flow and color
images. We have also compared the proposed GestFormer in terms of resource
efficiency and number of operations. The source code is available at
this https URL

我们提出了一种新颖的 GestFormer 架构，用于动态手势识别，旨在提供一种资源高效的 Transformer 模型，提高姿势的细节性能和资源效率。