This contribution presents a deep learning method for the extraction and
fusion of information relating to kidney stone fragments acquired from
different viewpoints of the endoscope. Surface and section fragment images are
jointly used during the training of the classifier to improve the
discrimination power of the features by adding attention layers at the end of
each convolutional block. This approach is specifically designed to mimic the
morpho-constitutional analysis performed in ex-vivo by biologists to visually
identify kidney stones by inspecting both views. The addition of attention
mechanisms to the backbone improved the results of single view extraction
backbones by 4% on average. Moreover, in comparison to the state-of-the-art,
the fusion of the deep features improved the overall results up to 11% in terms
of kidney stone classification accuracy.

本研究提出了一种基于深度学习的方法，以从内窥镜不同视角获取的肾结石碎片相关信息。通过在每个卷积块的末端添加注意力层，联合使用表面和切面碎片图像来训练分类器，从而改善特征的区分能力，进而模拟进行这种形态构成分析的方法，与此同时，在深度特征的融合上，本方法相对于现有技术提高了最多 11％的肾结石分类准确性。

通过注意力和多视角特征融合策略提高肾结石识别

Improved Kidney Stone Recognition Through Attention and Multi-View Feature Fusion Strategies

Flexible endoscopes for colonoscopy present several limitations due to their
inherent complexity, resulting in patient discomfort and lack of intuitiveness
for clinicians. Robotic devices together with autonomous control represent a
viable solution to reduce the workload of endoscopists and the training time
while improving the overall procedure outcome. Prior works on autonomous
endoscope control use heuristic policies that limit their generalisation to the
unstructured and highly deformable colon environment and require frequent human
intervention. This work proposes an image-based control of the endoscope using
Deep Reinforcement Learning, called Deep Visuomotor Control (DVC), to exhibit
adaptive behaviour in convoluted sections of the colon tract. DVC learns a
mapping between the endoscopic images and the control signal of the endoscope.
A first user study of 20 expert gastrointestinal endoscopists was carried out
to compare their navigation performance with DVC policies using a realistic
virtual simulator. The results indicate that DVC shows equivalent performance
on several assessment parameters, being more safer. Moreover, a second user
study with 20 novice participants was performed to demonstrate easier human
supervision compared to a state-of-the-art heuristic control policy. Seamless
supervision of colonoscopy procedures would enable interventionists to focus on
the medical decision rather than on the control problem of the endoscope.

本研究通过使用深度强化学习提出一种基于图像的内窥镜控制方法，能在大肠中出现复杂的部位展现出适应性行为，相比现有启发式控制策略在评估参数上表现相当甚至更安全，并且展示出更易于人类监督的特点，这将使介入医生能够将注意力集中在医疗决策上而不是内镜控制问题上。