Recognizing and comprehending human actions and gestures is a crucial
perception requirement for robots to interact with humans and carry out tasks
in diverse domains, including service robotics, healthcare, and manufacturing.
Event cameras, with their ability to capture fast-moving objects at a high
temporal resolution, offer new opportunities compared to standard action
recognition in RGB videos. However, previous research on event camera action
recognition has primarily focused on sensor-specific network architectures and
image encoding, which may not be suitable for new sensors and limit the use of
recent advancements in transformer-based architectures. In this study, we
employ a computationally efficient model, namely the video transformer network
(VTN), which initially acquires spatial embeddings per event-frame and then
utilizes a temporal self-attention mechanism. In order to better adopt the VTN
for the sparse and fine-grained nature of event data, we design
Event-Contrastive Loss ($\mathcal{L}_{EC}$) and event-specific augmentations.
Proposed $\mathcal{L}_{EC}$ promotes learning fine-grained spatial cues in the
spatial backbone of VTN by contrasting temporally misaligned frames. We
evaluate our method on real-world action recognition of N-EPIC Kitchens
dataset, and achieve state-of-the-art results on both protocols - testing in
seen kitchen (\textbf{74.9\%} accuracy) and testing in unseen kitchens
(\textbf{42.43\% and 46.66\% Accuracy}). Our approach also takes less
computation time compared to competitive prior approaches, which demonstrates
the potential of our framework \textit{EventTransAct} for real-world
applications of event-camera based action recognition. Project Page:
https://tristandb8.github.io/EventTransAct_webpage/

本研究旨在通过使用事件相机进行行为识别，提出了一种基于视频变换网络和事件对比损失的计算高效模型。该模型在真实世界中的行为识别任务中表现出卓越性能，并且相较于其他方法具有更低的计算时间，适用于基于事件相机的行为识别应用。