As part of their training all medical students and residents have to pass
basic surgical tasks such as knot tying, needle-passing, and suturing. Their
assessment is typically performed in the operating room by surgical faculty
where mistakes and failure by the student increases the operation time and
cost. This evaluation is quantitative and has a low margin of error. Simulation
has emerged as a cost effective option but it lacks assessment or requires
additional expensive hardware for evaluation. Apps that provide training videos
on surgical knot trying are available to students but none have evaluation. We
propose a cascaded neural network architecture that evaluates a student's
performance just from a video of themselves simulating a surgical knot tying
task. Our model converts video frame images into feature vectors with a
pre-trained deep convolutional network and then models the sequence of frames
with a temporal network. We obtained videos of medical students and residents
from the Robert Wood Johnson Hospital performing knot tying on a standardized
simulation kit. We manually annotated each video and proceeded to do a
five-fold cross-validation study on them. Our model achieves a median
precision, recall, and F1-score of 0.71, 0.66, and 0.65 respectively in
determining the level of knot related tasks of tying and pushing the knot. Our
mean precision score averaged across different probability thresholds is 0.8.
Both our F1-score and mean precision score are 8% and 30% higher than that of a
recently published study for the same problem. We expect the accuracy of our
model to further increase as we add more training videos to the model thus
making it a practical solution that students can use to evaluate themselves.

我们提出了一种级联神经网络架构，仅通过学生模拟外科结扎任务的视频评估其表现，通过预训练的深度卷积网络将视频帧图像转换为特征向量，然后用时间网络对帧序列建模。在模拟结扎实验中，我们的模型在决定绳结相关任务的水平时实现了 0.71 的中位数准确率、0.66 的中位数召回率和 0.65 的中位数 F1 分数。

一个级联神经网络系统用于评估外科手结扎模拟中学生表现

A Cascaded Neural Network System For Rating Student Performance In  Surgical Knot Tying Simulation

In this paper, we approach the challenging problem of motion planning for
knot tying. We propose a hierarchical approach in which the top layer produces
a topological plan and the bottom layer translates this plan into continuous
robot motion. The top layer decomposes a knotting task into sequences of
abstract topological actions based on knot theory. The bottom layer translates
each of these abstract actions into robot motion trajectories through learned
topological motion primitives. To adapt each topological action to the specific
rope geometry, the motion primitives take the observed rope configuration as
input. We train the motion primitives by imitating human demonstrations and
reinforcement learning in simulation. To generalize human demonstrations of
simple knots into more complex knots, we observe similarities in the motion
strategies of different topological actions and design the neural network
structure to exploit such similarities. We demonstrate that our learned motion
primitives can be used to efficiently generate motion plans for tying the
overhand knot. The motion plan can then be executed on a real robot using
visual tracking and Model Predictive Control. We also demonstrate that our
learned motion primitives can be composed to tie a more complex pentagram-like
knot despite being only trained on human demonstrations of simpler knots.

本文利用层次结构法解决了结绳节点的运动规划问题，使用拓扑学理论将结绳任务分解成抽象的拓扑动作，并通过学习的拓扑运动基元将每个抽象动作转化为机器人运动轨迹。通过仿真中的人类演示和强化学习来训练这些动作，最终成功地将这些学习到的动作组合起来将复杂的 “五角星” 状结绳节点进行绑定。