We present a new approach for vision-based force estimation in Minimally
Invasive Robotic Surgery based on frequency domain basis of motion of organs
derived directly from video. Using internal movements generated by natural
processes like breathing or the cardiac cycle, we infer the image-space basis
of the motion on the frequency domain. As we are working with this
representation, we discretize the problem to a limited amount of
low-frequencies to build an image-space mechanical model of the environment. We
use this pre-built model to define our force estimation problem as a dynamic
constraint problem. We demonstrate that this method can estimate point contact
forces reliably for silicone phantom and ex-vivo experiments, matching real
readings from a force sensor. In addition, we perform qualitative experiments
in which we synthesize coherent force textures from surgical videos over a
certain region of interest selected by the user. Our method demonstrates good
results for both quantitative and qualitative analysis, providing a good
starting point for a purely vision-based method for surgical force estimation.

基于频域动态基础的图像的方法，用于在微创机器人手术中进行视觉下的力量估计。应用自然过程中产生的内部运动来推断频域上图像空间中的运动基础，并将问题离散化为有限数量的低频，以构建环境的图像空间机械模型。该方法通过硅仿体和离体实验可靠地估计点接触力，并与力传感器的实际读数相匹配。此外，通过合成手术视频的连贯力纹理，在用户选择的感兴趣区域进行定性实验。本方法在定量和定性分析方面表现出良好的结果，为纯粹基于视觉的外科手术力量估计提供了良好的起点。

SurgeMOD：将图像空间组织运动转化为基于视觉的手术力量

SurgeMOD: Translating image-space tissue motions into vision-based  surgical forces

Precisely determining the contact force during safe interaction in Minimally
Invasive Robotic Surgery (MIRS) is still an open research challenge. Inspired
by post-operative qualitative analysis from surgical videos, the use of
cross-modality data driven deep neural network models has been one of the
newest approaches to predict sensorless force trends. However, these methods
required for large and variable datasets which are not currently available. In
this paper, we present a new vision-haptic dataset (DaFoEs) with variable soft
environments for the training of deep neural models. In order to reduce the
bias from a single dataset, we present a pipeline to generalize different
vision and state data inputs for mixed dataset training, using a previously
validated dataset with different setup. Finally, we present a variable
encoder-decoder architecture to predict the forces done by the laparoscopic
tool using single input or sequence of inputs. For input sequence, we use a
recurrent decoder, named with the prefix R, and a new temporal sampling to
represent the acceleration of the tool. During our training, we demonstrate
that single dataset training tends to overfit to the training data domain, but
has difficulties on translating the results across new domains. However,
dataset mixing presents a good translation with a mean relative estimated force
error of 5% and 12% for the recurrent and non-recurrent models respectively.
Our method, also marginally increase the effectiveness of transformers for
force estimation up to a maximum of ~15%, as the volume of available data is
increase by 150%. In conclusion, we demonstrate that mixing experimental set
ups for vision-state force estimation in MIRS is a possible approach towards
the general solution of the problem.

通过创建新的视触觉数据集，使用深度神经网络模型进行训练，并将不同的视觉和状态数据输入泛化为混合数据集训练的通用管线，该研究展示了在微创机器人手术中，混合实验设置用于视觉状态力估计是解决问题的一个可行方法。