In various applications, such as virtual reality and gaming, simulating the
deformation of soft tissues in the human body during interactions with external
objects is essential. Traditionally, Finite Element Methods (FEM) have been
employed for this purpose, but they tend to be slow and resource-intensive. In
this paper, we propose a unified representation of human body shape and soft
tissue with a data-driven simulator of non-rigid deformations. This approach
enables rapid simulation of realistic interactions.
Our method builds upon the SMPL model, which generates human body shapes
considering rigid transformations. We extend SMPL by incorporating a soft
tissue layer and an intuitive representation of external forces applied to the
body during object interactions. Specifically, we mapped the 3D body shape and
soft tissue and applied external forces to 2D UV maps. Leveraging a UNET
architecture designed for 2D data, our approach achieves high-accuracy
inference in real time. Our experiment shows that our method achieves plausible
deformation of the soft tissue layer, even for unseen scenarios.

人体软组织的非刚性变形模拟是在虚拟现实和游戏等应用中至关重要的，传统的有限元方法 (FEM) 过于缓慢和资源密集，本文提出了一种基于数据驱动的非刚性变形模拟器的统一表示方法，能够实现快速仿真真实交互。

STMPL：人体软组织模拟

STMPL: Human Soft-Tissue Simulation

Collecting real-world data is often considered the bottleneck of Artificial
Intelligence, stalling the research progress in several fields, one of which is
camera localization. End-to-end camera localization methods are still
outperformed by traditional methods, and we argue that the inconsistencies
associated with the data collection techniques are restraining the potential of
end-to-end methods. Inspired by the recent data-centric paradigm, we propose a
framework that synthesizes large localization datasets based on realistic 3D
reconstructions of the real world. Our framework, termed Synfeal: Synthetic
from Real, is an open-source, data-driven simulator that synthesizes RGB images
by moving a virtual camera through a realistic 3D textured mesh, while
collecting the corresponding ground-truth camera poses. The results validate
that the training of camera localization algorithms on datasets generated by
Synfeal leads to better results when compared to datasets generated by
state-of-the-art methods. Using Synfeal, we conducted the first analysis of the
relationship between the size of the dataset and the performance of camera
localization algorithms. Results show that the performance significantly
increases with the dataset size. Our results also suggest that when a large
localization dataset with high quality is available, training from scratch
leads to better performances. Synfeal is publicly available at
this https URL

本研究中，我们提出了一种名为 Synfeal 的数据驱动模拟器，通过在真实世界的逼真三维重建中移动虚拟摄像机并收集相应的地面真实摄像机姿势来合成真实场景的大型定位数据集。通过使用 Synfeal 合成数据集并对其进行有针对性的训练，本文证明了这能够优于其他数据集的训练效果，而且当数据集的大小越大时，算法的性能也显著提高。

Synfeal: 基于数据驱动的端到端相机定位模拟器

Synfeal: A Data-Driven Simulator for End-to-End Camera Localization

In this work we are the first to present an offline policy gradient method
for learning imitative policies for complex urban driving from a large corpus
of real-world demonstrations. This is achieved by building a differentiable
data-driven simulator on top of perception outputs and high-fidelity HD maps of
the area. It allows us to synthesize new driving experiences from existing
demonstrations using mid-level representations. Using this simulator we then
train a policy network in closed-loop employing policy gradients. We train our
proposed method on 100 hours of expert demonstrations on urban roads and show
that it learns complex driving policies that generalize well and can perform a
variety of driving maneuvers. We demonstrate this in simulation as well as
deploy our model to self-driving vehicles in the real-world. Our method
outperforms previously demonstrated state-of-the-art for urban driving
scenarios -- all this without the need for complex state perturbations or
collecting additional on-policy data during training. We make code and data
publicly available.

本文提出了一种离线策略梯度方法，通过在感知输出和高保真度地图上构建可微分的数据驱动模拟器来从现实世界的大量演示中学习城市驾驶的模仿策略，进而在闭环中使用策略梯度训练策略网络，并通过在 100 小时的专家演示上进行训练，展示其能够学习出高泛化性能和多种驾驶动作的城市驾驶策略，并且无需进行复杂的状态扰动或采集进一步的在线策略数据而在真实世界中优于现有的最新城市驾驶方案。