As consumer Virtual Reality (VR) and Mixed Reality (MR) technologies gain
momentum, there's a growing focus on the development of engagements with 3D
virtual content. Unfortunately, traditional techniques for content creation,
editing, and interaction within these virtual spaces are fraught with
difficulties. They tend to be not only engineering-intensive but also require
extensive expertise, which adds to the frustration and inefficiency in virtual
object manipulation. Our proposed VR-GS system represents a leap forward in
human-centered 3D content interaction, offering a seamless and intuitive user
experience. By developing a physical dynamics-aware interactive Gaussian
Splatting in a Virtual Reality setting, and constructing a highly efficient
two-level embedding strategy alongside deformable body simulations, VR-GS
ensures real-time execution with highly realistic dynamic responses. The
components of our Virtual Reality system are designed for high efficiency and
effectiveness, starting from detailed scene reconstruction and object
segmentation, advancing through multi-view image in-painting, and extending to
interactive physics-based editing. The system also incorporates real-time
deformation embedding and dynamic shadow casting, ensuring a comprehensive and
engaging virtual experience.Our project page is available at:
this https URL

消费者虚拟现实（VR）和混合现实（MR）技术的发展引发了对与 3D 虚拟内容的互动的关注，然而传统的内容创作、编辑和交互技术存在困难。本研究提出了 VR-GS 系统，通过在虚拟现实环境中开发具有物理动力学感知的交互高斯碎片技术，并结合可变形物体的模拟，以实现无缝和直观的用户体验。该系统通过详细的场景重建和物体分割、多视角图像修复，以及基于物理的交互编辑，同时包含实时变形嵌入和动态阴影投射等功能，确保了全面且引人入胜的虚拟体验。