Despite significant advancements in robotic systems and surgical data
science, ensuring safe and optimal execution in robot-assisted minimally
invasive surgery (RMIS) remains a complex challenge. Current surgical error
detection methods involve two parts: identifying surgical gestures and then
detecting errors within each gesture clip. These methods seldom consider the
rich contextual and semantic information inherent in surgical videos, limiting
their performance due to reliance on accurate gesture identification. Motivated
by the chain-of-thought prompting in natural language processing, this letter
presents a novel and real-time end-to-end error detection framework,
Chain-of-Thought (COG) prompting, leveraging contextual information from
surgical videos. This encompasses two reasoning modules designed to mimic the
decision-making processes of expert surgeons. Concretely, we first design a
Gestural-Visual Reasoning module, which utilizes transformer and attention
architectures for gesture prompting, while the second, a Multi-Scale Temporal
Reasoning module, employs a multi-stage temporal convolutional network with
both slow and fast paths for temporal information extraction. We extensively
validate our method on the public benchmark RMIS dataset JIGSAWS. Our method
encapsulates the reasoning processes inherent to surgical activities enabling
it to outperform the state-of-the-art by 4.6% in F1 score, 4.6% in Accuracy,
and 5.9% in Jaccard index while processing each frame in 6.69 milliseconds on
average, demonstrating the great potential of our approach in enhancing the
safety and efficacy of RMIS procedures and surgical education. The code will be
available.

本论文提出了一种新颖的实时端到端错误检测框架，称为 Chain-of-Thought (COG) prompting，利用手术视频中的上下文信息来提高机器人辅助微创手术的安全性和效果。通过使用 transformer、attention 和多阶段时序卷积网络等方法，该方法在公共基准 RMIS 数据集 JIGSAWS 上优于目前最先进方法的 F1 分数、准确率和 Jaccard 指数，并且平均每帧处理时间为 6.69 毫秒。