Immersive authoring provides an intuitive medium for users to create 3D scenes via direct manipulation in Virtual Reality (VR). Recent advances in generative AI have enabled the automatic creation of realistic 3D layouts. However, it is unclear how capabilities of generative AI can be used in immersive authoring to support fluid interactions, user agency, and creativity. We introduce VRCopilot, a mixed-initiative system that integrates pre-trained generative AI models into immersive authoring to facilitate human-AI co-creation in VR. VRCopilot presents multimodal interactions to support rapid prototyping and iterations with AI, and intermediate representations such as wireframes to augment user controllability over the created content. Through a series of user studies, we evaluated the potential and challenges in manual, scaffolded, and automatic creation in immersive authoring. We found that scaffolded creation using wireframes enhanced the user agency compared to automatic creation. We also found that manual creation via multimodal specification offers the highest sense of creativity and agency.

本研究解决了如何将生成性人工智能的能力应用于虚拟现实中的沉浸式创作，以支持流畅的互动和用户创造力的不足。我们提出的VRCopilot系统结合了预训练的生成性AI模型，通过多模态互动和中介表达方式增强用户对创建内容的可控性。研究发现，使用线框进行的辅助创作比完全自动生成的方式能够提升用户的主动性，而通过多模态规格进行的手动创作则提供了最高的创造感和自主性。

VRCopilot：在虚拟现实中利用生成性人工智能模型创作3D布局