For robot-assisted surgery, an accurate surgical report reflects clinical
operations during surgery and helps document entry tasks, post-operative
analysis and follow-up treatment. It is a challenging task due to many complex
and diverse interactions between instruments and tissues in the surgical scene.
Although existing surgical report generation methods based on deep learning
have achieved large success, they often ignore the interactive relation between
tissues and instrumental tools, thereby degrading the report generation
performance. This paper presents a neural network to boost surgical report
generation by explicitly exploring the interactive relation between tissues and
surgical instruments. We validate the effectiveness of our method on a
widely-used robotic surgery benchmark dataset, and experimental results show
that our network can significantly outperform existing state-of-the-art
surgical report generation methods (e.g., 7.48% and 5.43% higher for BLEU-1 and
ROUGE).

本研究提出了一种神经网络方法，通过明确探索组织和手术器械之间的互动关系来提高手术报告的生成，实验结果表明我们的方法可以显著优于现有最先进的手术报告生成方法。

通过场景图学习捕捉动态交互关系，用于机器人手术报告生成

Dynamic Interactive Relation Capturing via Scene Graph Learning for  Robotic Surgical Report Generation

Generating surgical reports aimed at surgical scene understanding in
robot-assisted surgery can contribute to documenting entry tasks and
post-operative analysis. Despite the impressive outcome, the deep learning
model degrades the performance when applied to different domains encountering
domain shifts. In addition, there are new instruments and variations in
surgical tissues appeared in robotic surgery. In this work, we propose
class-incremental domain adaptation (CIDA) with a multi-layer transformer-based
model to tackle the new classes and domain shift in the target domain to
generate surgical reports during robotic surgery. To adapt incremental classes
and extract domain invariant features, a class-incremental (CI) learning method
with supervised contrastive (SupCon) loss is incorporated with a feature
extractor. To generate caption from the extracted feature, curriculum by
one-dimensional gaussian smoothing (CBS) is integrated with a multi-layer
transformer-based caption prediction model. CBS smoothes the features embedding
using anti-aliasing and helps the model to learn domain invariant features. We
also adopt label smoothing (LS) to calibrate prediction probability and obtain
better feature representation with both feature extractor and captioning model.
The proposed techniques are empirically evaluated by using the datasets of two
surgical domains, such as nephrectomy operations and transoral robotic surgery.
We observe that domain invariant feature learning and the well-calibrated
network improves the surgical report generation performance in both source and
target domain under domain shift and unseen classes in the manners of one-shot
and few-shot learning. The code is publicly available at
this https URL

提出了一种基于多层变压器的模型的分类增量域自适应（CIDA）方法，用于在机器人手术过程中生成手术报告，并对其进行了实证评估。