In the realm of automated robotic surgery and computer-assisted
interventions, understanding robotic surgical activities stands paramount.
Existing algorithms dedicated to surgical activity recognition predominantly
cater to pre-defined closed-set paradigms, ignoring the challenges of
real-world open-set scenarios. Such algorithms often falter in the presence of
test samples originating from classes unseen during training phases. To tackle
this problem, we introduce an innovative Open-Set Surgical Activity Recognition
(OSSAR) framework. Our solution leverages the hyperspherical reciprocal point
strategy to enhance the distinction between known and unknown classes in the
feature space. Additionally, we address the issue of over-confidence in the
closed set by refining model calibration, avoiding misclassification of unknown
classes as known ones. To support our assertions, we establish an open-set
surgical activity benchmark utilizing the public JIGSAWS dataset. Besides, we
also collect a novel dataset on endoscopic submucosal dissection for surgical
activity tasks. Extensive comparisons and ablation experiments on these
datasets demonstrate the significant outperformance of our method over existing
state-of-the-art approaches. Our proposed solution can effectively address the
challenges of real-world surgical scenarios. Our code is publicly accessible at
this https URL

在自动机器人手术和计算机辅助干预领域中，理解机器人手术活动至关重要。为了解决真实世界中的开放场景挑战，我们引入了一种创新的开集手术活动识别（OSSAR）框架，通过使用超球面互补点策略增强特征空间中已知和未知类之间的差异，并通过改进模型校准来避免未知类被误分类为已知类。通过公开的 JIGSAWS 数据集以及我们自己收集的内窥镜黏膜下剥离的手术活动数据集，我们对我们的方法进行了广泛的对比和割离实验，结果表明我们的方法在现有最先进方法上取得了显著的提升。我们的方案可以有效处理真实世界手术场景的挑战。我们的代码可以在此 https URL 上公开访问。

面向机器人辅助手术的开放式外科活动识别

OSSAR: Towards Open-Set Surgical Activity Recognition in Robot-assisted  Surgery

Most studies on surgical activity recognition utilizing Artificial
intelligence (AI) have focused mainly on recognizing one type of activity from
small and mono-centric surgical video datasets. It remains speculative whether
those models would generalize to other centers. In this work, we introduce a
large multi-centric multi-activity dataset consisting of 140 videos
(MultiBypass140) of laparoscopic Roux-en-Y gastric bypass (LRYGB) surgeries
performed at two medical centers: the University Hospital of Strasbourg
(StrasBypass70) and Inselspital, Bern University Hospital (BernBypass70). The
dataset has been fully annotated with phases and steps. Furthermore, we assess
the generalizability and benchmark different deep learning models in 7
experimental studies: 1) Training and evaluation on BernBypass70; 2) Training
and evaluation on StrasBypass70; 3) Training and evaluation on the
MultiBypass140; 4) Training on BernBypass70, evaluation on StrasBypass70; 5)
Training on StrasBypass70, evaluation on BernBypass70; Training on
MultiBypass140, evaluation 6) on BernBypass70 and 7) on StrasBypass70. The
model's performance is markedly influenced by the training data. The worst
results were obtained in experiments 4) and 5) confirming the limited
generalization capabilities of models trained on mono-centric data. The use of
multi-centric training data, experiments 6) and 7), improves the generalization
capabilities of the models, bringing them beyond the level of independent
mono-centric training and validation (experiments 1) and 2)). MultiBypass140
shows considerable variation in surgical technique and workflow of LRYGB
procedures between centers. Therefore, generalization experiments demonstrate a
remarkable difference in model performance. These results highlight the
importance of multi-centric datasets for AI model generalization to account for
variance in surgical technique and workflows.

通过评估多中心多活动数据集在七个实验研究中的表现，该论文展示了使用人工智能模型进行手术活动识别的一种方法，强调使用多中心数据集来提高模型的泛化能力。

多中心泛化中的挑战：Roux-en-Y 胃旁路手术中的阶段和步骤识别

Challenges in Multi-centric Generalization: Phase and Step Recognition  in Roux-en-Y Gastric Bypass Surgery

Surgical activity recognition and prediction can help provide important
context in many Robot-Assisted Surgery (RAS) applications, for example,
surgical progress monitoring and estimation, surgical skill evaluation, and
shared control strategies during teleoperation. Transformer models were first
developed for Natural Language Processing (NLP) to model word sequences and
soon the method gained popularity for general sequence modeling tasks. In this
paper, we propose the novel use of a Transformer model for three tasks: gesture
recognition, gesture prediction, and trajectory prediction during RAS. We
modify the original Transformer architecture to be able to generate the current
gesture sequence, future gesture sequence, and future trajectory sequence
estimations using only the current kinematic data of the surgical robot
end-effectors. We evaluate our proposed models on the JHU-ISI Gesture and Skill
Assessment Working Set (JIGSAWS) and use Leave-One-User-Out (LOUO)
cross-validation to ensure the generalizability of our results. Our models
achieve up to 89.3\% gesture recognition accuracy, 84.6\% gesture prediction
accuracy (1 second ahead) and 2.71mm trajectory prediction error (1 second
ahead). Our models are comparable to and able to outperform state-of-the-art
methods while using only the kinematic data channel. This approach can enable
near-real time surgical activity recognition and prediction.

本篇论文提出了一种新的方法，基于改进的 Transformer 模型，利用手术机器人端效应器当前的运动数据，可以实现手势识别、手势预测、轨迹预测等三个任务，能够实现近乎实时的手术活动识别和预测，实验结果表明该方案可以比同类方法表现更好。