The increasing demand for virtual reality applications has highlighted the
significance of crafting immersive 3D assets. We present a text-to-3D
360$^{\circ}$ scene generation pipeline that facilitates the creation of
comprehensive 360$^{\circ}$ scenes for in-the-wild environments in a matter of
minutes. Our approach utilizes the generative power of a 2D diffusion model and
prompt self-refinement to create a high-quality and globally coherent panoramic
image. This image acts as a preliminary "flat" (2D) scene representation.
Subsequently, it is lifted into 3D Gaussians, employing splatting techniques to
enable real-time exploration. To produce consistent 3D geometry, our pipeline
constructs a spatially coherent structure by aligning the 2D monocular depth
into a globally optimized point cloud. This point cloud serves as the initial
state for the centroids of 3D Gaussians. In order to address invisible issues
inherent in single-view inputs, we impose semantic and geometric constraints on
both synthesized and input camera views as regularizations. These guide the
optimization of Gaussians, aiding in the reconstruction of unseen regions. In
summary, our method offers a globally consistent 3D scene within a
360$^{\circ}$ perspective, providing an enhanced immersive experience over
existing techniques. Project website at: this http URL

我们提供了一个文本转 3D 360 度场景生成流水线，能够在几分钟内为野外环境创建综合的 360 度场景。我们的方法利用 2D 扩散模型的生成能力和提示自我完善来创建高质量和全局连贯的全景图像，这作为一个初步的 “平面”（2D）场景表示。随后，通过采用粒子技术将它提升到 3D 高斯函数，以实现实时浏览。为了产生一致的 3D 几何结构，我们的流水线通过将 2D 单目深度对齐成全局优化点云，构建了一个空间连贯的结构。这个点云作为 3D 高斯函数的初始状态的质心。为了解决单视角输入固有的不可见问题，我们对合成和输入相机视图应用语义和几何约束作为规范，这些约束指导高斯函数的优化，帮助重建不可见的区域。总之，我们的方法提供了一个全局一致的 360 度视角的 3D 场景，相较于现有技术提供了更加增强的沉浸式体验。项目网址：this http URL

DreamScene360: 无约束的文本到 3D 场景生成与全景高斯喷洒

DreamScene360: Unconstrained Text-to-3D Scene Generation with Panoramic  Gaussian Splatting

Image-based rendering techniques stand at the core of an immersive experience
for the user, as they generate novel views given a set of multiple input
images. Since they have shown good performance in terms of objective and
subjective quality, the research community devotes great effort to their
improvement. However, the large volume of data necessary to render at the
receiver's side hinders applications in limited bandwidth environments or
prevents their employment in real-time applications. We present LeHoPP, a
method for input pixel pruning, where we examine the importance of each input
pixel concerning the rendered view, and we avoid the use of irrelevant pixels.
Even without retraining the image-based rendering network, our approach shows a
good trade-off between synthesis quality and pixel rate. When tested in the
general neural rendering framework, compared to other pruning baselines, LeHoPP
gains between $0.9$ dB and $3.6$ dB on average.

本研究提出了一种名为 'LeHoPP' 的像素精简方法，可以提高基于图像的渲染网络的性能，从而在低带宽环境或实时应用中提高渲染速度和质量。

学习如何为多视角神经图像合成修剪像素

Learn how to Prune Pixels for Multi-view Neural Image-based Synthesis

Along with the massive growth of the Internet from the 1990s until now,
various innovative technologies have been created to bring users breathtaking
experiences with more virtual interactions in cyberspace. Many virtual
environments with thousands of services and applications, from social networks
to virtual gaming worlds, have been developed with immersive experience and
digital transformation, but most are incoherent instead of being integrated
into a platform. In this context, metaverse, a term formed by combining meta
and universe, has been introduced as a shared virtual world that is fueled by
many emerging technologies, such as fifth-generation networks and beyond,
virtual reality, and artificial intelligence (AI). Among such technologies, AI
has shown the great importance of processing big data to enhance immersive
experience and enable human-like intelligence of virtual agents. In this
survey, we make a beneficial effort to explore the role of AI in the foundation
and development of the metaverse. We first deliver a preliminary of AI,
including machine learning algorithms and deep learning architectures, and its
role in the metaverse. We then convey a comprehensive investigation of AI-based
methods concerning six technical aspects that have potentials for the
metaverse: natural language processing, machine vision, blockchain, networking,
digital twin, and neural interface, and being potential for the metaverse.
Subsequently, several AI-aided applications, such as healthcare, manufacturing,
smart cities, and gaming, are studied to be deployed in the virtual worlds.
Finally, we conclude the key contribution of this survey and open some future
research directions in AI for the metaverse.

本文调查了 AI 在元宇宙（metaverse）的基础和发展中所扮演的角色，从自然语言处理，机器视觉，区块链，网络，数字孪生和神经接口等六个技术方面探索了 AI 在元宇宙中的潜力，并介绍了几个使用人工智能的应用，包括医疗保健，制造业，智慧城市和游戏等。