Automated surgical step recognition is an important task that can
significantly improve patient safety and decision-making during surgeries.
Existing state-of-the-art methods for surgical step recognition either rely on
separate, multi-stage modeling of spatial and temporal information or operate
on short-range temporal resolution when learned jointly. However, the benefits
of joint modeling of spatio-temporal features and long-range information are
not taken in account. In this paper, we propose a vision transformer-based
approach to jointly learn spatio-temporal features directly from sequence of
frame-level patches. Our method incorporates a gated-temporal attention
mechanism that intelligently combines short-term and long-term spatio-temporal
feature representations. We extensively evaluate our approach on two cataract
surgery video datasets, namely Cataract-101 and D99, and demonstrate superior
performance compared to various state-of-the-art methods. These results
validate the suitability of our proposed approach for automated surgical step
recognition. Our code is released at:
this https URL

提出了一种基于视觉转换器的方法来联合学习时空特征，以自动进行手术步骤识别，通过在两个白内障手术视频数据集上的广泛评估，证明了该方法在自动手术步骤识别方面具有优异的性能。