Creating scenes for captured motions that achieve realistic human-scene
interaction is crucial for 3D animation in movies or video games. As character
motion is often captured in a blue-screened studio without real furniture or
objects in place, there may be a discrepancy between the planned motion and the
captured one. This gives rise to the need for automatic scene layout generation
to relieve the burdens of selecting and positioning furniture and objects.
Previous approaches cannot avoid artifacts like penetration and floating due to
the lack of physical constraints. Furthermore, some heavily rely on specific
data to learn the contact affordances, restricting the generalization ability
to different motions. In this work, we present a physics-based approach that
simultaneously optimizes a scene layout generator and simulates a moving human
in a physics simulator. To attain plausible and realistic interaction motions,
our method explicitly introduces physical constraints. To automatically recover
and generate the scene layout, we minimize the motion tracking errors to
identify the objects that can afford interaction. We use reinforcement learning
to perform a dual-optimization of both the character motion imitation
controller and the scene layout generator. To facilitate the optimization, we
reshape the tracking rewards and devise pose prior guidance obtained from our
estimated pseudo-contact labels. We evaluate our method using motions from SAMP
and PROX, and demonstrate physically plausible scene layout reconstruction
compared with the previous kinematics-based method.

本研究提出了基于物理原理的方法，同时优化场景布局生成器和物理模拟器中的人体移动，以实现可信和真实的交互动作，通过最小化运动追踪误差以自动恢复和生成场景布局，并使用强化学习进行双重优化，通过重新塑造追踪奖励和估计的伪接触标签获得的姿势先验指导来促进优化，使用 SAMP 和 PROX 的动作进行评估，并与基于运动学的先前方法进行比较，展示了物理上可信的场景布局重建。

基于物理的从人体动作生成场景布局

Physics-based Scene Layout Generation from Human Motion

Realistic object interactions are crucial for creating immersive virtual
experiences, yet synthesizing realistic 3D object dynamics in response to novel
interactions remains a significant challenge. Unlike unconditional or
text-conditioned dynamics generation, action-conditioned dynamics requires
perceiving the physical material properties of objects and grounding the 3D
motion prediction on these properties, such as object stiffness. However,
estimating physical material properties is an open problem due to the lack of
material ground-truth data, as measuring these properties for real objects is
highly difficult. We present PhysDreamer, a physics-based approach that endows
static 3D objects with interactive dynamics by leveraging the object dynamics
priors learned by video generation models. By distilling these priors,
PhysDreamer enables the synthesis of realistic object responses to novel
interactions, such as external forces or agent manipulations. We demonstrate
our approach on diverse examples of elastic objects and evaluate the realism of
the synthesized interactions through a user study. PhysDreamer takes a step
towards more engaging and realistic virtual experiences by enabling static 3D
objects to dynamically respond to interactive stimuli in a physically plausible
manner. See our project page at this https URL

通过基于物理原理的方法，我们提出了 PhysDreamer，它能够使静态的 3D 对象能够以虚拟实境中真实的方式动态地响应交互刺激，实现了更加引人入胜和逼真的虚拟体验。

PhysDreamer: 基于物理的通过视频生成与 3D 物体交互

PhysDreamer: Physics-Based Interaction with 3D Objects via Video  Generation

Lidar-based object detectors are critical parts of the 3D perception pipeline
in autonomous navigation systems such as self-driving cars. However, they are
known to be sensitive to adverse weather conditions such as rain, snow and fog
due to reduced signal-to-noise ratio (SNR) and signal-to-background ratio
(SBR). As a result, lidar-based object detectors trained on data captured in
normal weather tend to perform poorly in such scenarios. However, collecting
and labelling sufficient training data in a diverse range of adverse weather
conditions is laborious and prohibitively expensive. To address this issue, we
propose a physics-based approach to simulate lidar point clouds of scenes in
adverse weather conditions. These augmented datasets can then be used to train
lidar-based detectors to improve their all-weather reliability. Specifically,
we introduce a hybrid Monte-Carlo based approach that treats (i) the effects of
large particles by placing them randomly and comparing their back reflected
power against the target, and (ii) attenuation effects on average through
calculation of scattering efficiencies from the Mie theory and particle size
distributions. Retraining networks with this augmented data improves mean
average precision evaluated on real world rainy scenes and we observe greater
improvement in performance with our model relative to existing models from the
literature. Furthermore, we evaluate recent state-of-the-art detectors on the
simulated weather conditions and present an in-depth analysis of their
performance.

通过物理学模拟，提出一种蒙特卡罗混合方法用于模拟恶劣天气下的 LiDAR 点云数据，从而扩展所有天气条件下目标检测器的可靠性，并表明相对于现有模型，在现实世界的雨天场景中，模型的性能有了显著改进。