This paper studies the problem of estimating physical properties (system
identification) through visual observations. To facilitate geometry-aware
guidance in physical property estimation, we introduce a novel hybrid framework
that leverages 3D Gaussian representation to not only capture explicit shapes
but also enable the simulated continuum to deduce implicit shapes during
training. We propose a new dynamic 3D Gaussian framework based on motion
factorization to recover the object as 3D Gaussian point sets across different
time states. Furthermore, we develop a coarse-to-fine filling strategy to
generate the density fields of the object from the Gaussian reconstruction,
allowing for the extraction of object continuums along with their surfaces and
the integration of Gaussian attributes into these continuums. In addition to
the extracted object surfaces, the Gaussian-informed continuum also enables the
rendering of object masks during simulations, serving as implicit shape
guidance for physical property estimation. Extensive experimental evaluations
demonstrate that our pipeline achieves state-of-the-art performance across
multiple benchmarks and metrics. Additionally, we illustrate the effectiveness
of the proposed method through real-world demonstrations, showcasing its
practical utility. Our project page is at this https URL

通过视觉观察来估计物理特性 (系统辨识) 的问题，本论文介绍了一种新颖的混合框架，该框架利用三维高斯表示不仅能捕捉明确的形状，还能使模拟连续推导训练中的隐含形状。我们提出了一种基于运动分解的新型动态三维高斯框架，以在不同时间状态下恢复物体作为三维高斯点集。此外，我们开发了一种由粗到细的填充策略，从高斯重建生成物体的密度场，允许提取物体连续及其表面，并将高斯属性融入这些连续体中。除提取的物体表面外，高斯信息的连续体还能在模拟中渲染物体掩膜，为物理特性估计提供隐含形状引导。大量实验评估表明，我们的管线在多个基准和度量标准上达到了最先进的性能。另外，我们通过真实世界演示展示了所提方法的实用性。

基于高斯信息的连续方法用于物性识别与模拟

Gaussian-Informed Continuum for Physical Property Identification and  Simulation

The advent of generative radiance fields has significantly promoted the
development of 3D-aware image synthesis. The cumulative rendering process in
radiance fields makes training these generative models much easier since
gradients are distributed over the entire volume, but leads to diffused object
surfaces. In the meantime, compared to radiance fields occupancy
representations could inherently ensure deterministic surfaces. However, if we
directly apply occupancy representations to generative models, during training
they will only receive sparse gradients located on object surfaces and
eventually suffer from the convergence problem. In this paper, we propose
Generative Occupancy Fields (GOF), a novel model based on generative radiance
fields that can learn compact object surfaces without impeding its training
convergence. The key insight of GOF is a dedicated transition from the
cumulative rendering in radiance fields to rendering with only the surface
points as the learned surface gets more and more accurate. In this way, GOF
combines the merits of two representations in a unified framework. In practice,
the training-time transition of start from radiance fields and march to
occupancy representations is achieved in GOF by gradually shrinking the
sampling region in its rendering process from the entire volume to a minimal
neighboring region around the surface. Through comprehensive experiments on
multiple datasets, we demonstrate that GOF can synthesize high-quality images
with 3D consistency and simultaneously learn compact and smooth object
surfaces. Code, models, and demo videos are available at
this https URL

本文提出了一种名为 GOF 的新模型，它基于生成亮度场，可以在不妨碍其训练收敛的情况下学习紧凑的物体表面。GOF 的关键在于从亮度场的累积渲染到只以表面点为基础的渲染的特定转换。通过对多个数据集的全面实验，我们证明了 GOF 可以生成具有 3D 一致性的高质量图像，并同时学习到紧凑而光滑的物体表面。

基于生成式占位场的三维表面感知图像合成

Generative Occupancy Fields for 3D Surface-Aware Image Synthesis

Point clouds acquired from range scans are often sparse, noisy, and
non-uniform. This paper presents a new point cloud upsampling network called
PU-GAN, which is formulated based on a generative adversarial network (GAN), to
learn a rich variety of point distributions from the latent space and upsample
points over patches on object surfaces. To realize a working GAN network, we
construct an up-down-up expansion unit in the generator for upsampling point
features with error feedback and self-correction, and formulate a
self-attention unit to enhance the feature integration. Further, we design a
compound loss with adversarial, uniform and reconstruction terms, to encourage
the discriminator to learn more latent patterns and enhance the output point
distribution uniformity. Qualitative and quantitative evaluations demonstrate
the quality of our results over the state-of-the-arts in terms of distribution
uniformity, proximity-to-surface, and 3D reconstruction quality.

本文提出了一种基于生成对抗网络（GAN）模型的新的点云上采样网络 PU-GAN，能够从潜在空间中学习到丰富的点分布并通过误差反馈和自校正实现对物体表面上的点的上采样，并通过鉴别器学习更多潜在模式以增强输出点的分布均匀性。定量和定性评估表明，PU-GAN 在分布均匀性、与物体表面的接近度以及三维重建质量方面，优于同类方法。