3D patient body modeling is critical to the success of automated patient
positioning for smart medical scanning and operating rooms. Existing CNN-based
end-to-end patient modeling solutions typically require a) customized network
designs demanding large amount of relevant training data, covering extensive
realistic clinical scenarios (e.g., patient covered by sheets), which leads to
suboptimal generalizability in practical deployment, b) expensive 3D human
model annotations, i.e., requiring huge amount of manual effort, resulting in
systems that scale poorly. To address these issues, we propose a generic
modularized 3D patient modeling method consists of (a) a multi-modal keypoint
detection module with attentive fusion for 2D patient joint localization, to
learn complementary cross-modality patient body information, leading to
improved keypoint localization robustness and generalizability in a wide
variety of imaging (e.g., CT, MRI etc.) and clinical scenarios (e.g., heavy
occlusions); and (b) a self-supervised 3D mesh regression module which does not
require expensive 3D mesh parameter annotations to train, bringing immediate
cost benefits for clinical deployment. We demonstrate the efficacy of the
proposed method by extensive patient positioning experiments on both public and
clinical data. Our evaluation results achieve superior patient positioning
performance across various imaging modalities in real clinical scenarios.

通过提出一种通用模块化的三维患者建模方法，即多模式关键点检测模块与自监督三维网格回归模块的结合，在实际临床场景中实现了改善关键点定位的稳健性和概括性，并取得了优越的患者定位性能。