As the significance of simulation in medical care and intervention continues
to grow, it is anticipated that a simplified and low-cost platform can be set
up to execute personalized diagnoses and treatments. 3D Slicer can not only
perform medical image analysis and visualization but can also provide surgical
navigation and surgical planning functions. In this paper, we have chosen 3D
Slicer as our base platform and monocular cameras are used as sensors. Then, We
used the neural radiance fields (NeRF) algorithm to complete the 3D model
reconstruction of the human head. We compared the accuracy of the NeRF
algorithm in generating 3D human head scenes and utilized the MarchingCube
algorithm to generate corresponding 3D mesh models. The individual's head pose,
obtained through single-camera vision, is transmitted in real-time to the scene
created within 3D Slicer. The demonstrations presented in this paper include
real-time synchronization of transformations between the human head model in
the 3D Slicer scene and the detected head posture. Additionally, we tested a
scene where a tool, marked with an ArUco Maker tracked by a single camera,
synchronously points to the real-time transformation of the head posture. These
demos indicate that our methodology can provide a feasible real-time simulation
platform for nasopharyngeal swab collection or intubation.

我们使用 3D Slicer 作为基础平台，利用单目相机将人头进行 3D 模型重建，实时通过单目视觉传输头部姿态，并演示了利用 ArUco Marker 追踪工具实时指向头部姿态变化的可行性，为鼻咽拭子采集或插管等提供了一个可行的实时仿真平台。

通过单目摄像机实时进行航行和安全进入自然开口的头部姿态估计和三维神经表面重建

Head Pose Estimation and 3D Neural Surface Reconstruction via Monocular  Camera in situ for Navigation and Safe Insertion into Natural Openings

We propose a self-supervised approach for learning physics-based subspaces
for real-time simulation. Existing learning-based methods construct subspaces
by approximating pre-defined simulation data in a purely geometric way.
However, this approach tends to produce high-energy configurations, leads to
entangled latent space dimensions, and generalizes poorly beyond the training
set. To overcome these limitations, we propose a self-supervised approach that
directly minimizes the system's mechanical energy during training. We show that
our method leads to learned subspaces that reflect physical equilibrium
constraints, resolve overfitting issues of previous methods, and offer
interpretable latent space parameters.

我们提出一种自监督方法，用于学习基于物理的子空间进行实时模拟。通过近似预定义的模拟数据以纯几何方式构建子空间的现有基于学习的方法。然而，这种方法倾向于产生高能量配置，导致了纠缠的潜空间维度，并且在训练集之外的泛化能力较差。为了克服这些限制，我们提出了一种自监督方法，在训练期间直接最小化系统的机械能。我们证明了我们的方法导致了学习到反映物理平衡约束的子空间，解决了先前方法的过拟合问题，并提供可解释的潜空间参数。

神经模式：非线性模态子空间的自监督学习

Neural Modes: Self-supervised Learning of Nonlinear Modal Subspaces

Global urbanization has underscored the significance of urban microclimates
for human comfort, health, and building/urban energy efficiency. They
profoundly influence building design and urban planning as major environmental
impacts. Understanding local microclimates is essential for cities to prepare
for climate change and effectively implement resilience measures. However,
analyzing urban microclimates requires considering a complex array of outdoor
parameters within computational domains at the city scale over a longer period
than indoors. As a result, numerical methods like Computational Fluid Dynamics
(CFD) become computationally expensive when evaluating the impact of urban
microclimates. The rise of deep learning techniques has opened new
opportunities for accelerating the modeling of complex non-linear interactions
and system dynamics. Recently, the Fourier Neural Operator (FNO) has been shown
to be very promising in accelerating solving the Partial Differential Equations
(PDEs) and modeling fluid dynamic systems. In this work, we apply the FNO
network for real-time three-dimensional (3D) urban wind field simulation. The
training and testing data are generated from CFD simulation of the urban area,
based on the semi-Lagrangian approach and fractional stepping method to
simulate urban microclimate features for modeling large-scale urban problems.
Numerical experiments show that the FNO model can accurately reconstruct the
instantaneous spatial velocity field. We further evaluate the trained FNO model
on unseen data with different wind directions, and the results show that the
FNO model can generalize well on different wind directions. More importantly,
the FNO approach can make predictions within milliseconds on the graphics
processing unit, making real-time simulation of 3D dynamic urban microclimate
possible.

利用 Fourier 神经算子 (FNO) 网络以实时三维城市风场模拟，通过半拉格朗日方法和分数步进法模拟大规模城市问题中的城市微气候特征，数值实验显示 FNO 模型能够准确重构瞬时空间速度场，在不同风向的未见数据上也具有良好的泛化能力，更重要的是 FNO 方法能够在毫秒内进行预测，实现了三维动态城市微气候的实时模拟。