360$^\circ$ panoramas are extensively utilized as environmental light sources
in computer graphics. However, capturing a 360$^\circ$ $\times$ 180$^\circ$
panorama poses challenges due to the necessity of specialized and costly
equipment, and additional human resources. Prior studies develop various
learning-based generative methods to synthesize panoramas from a single Narrow
Field-of-View (NFoV) image, but they are limited in alterable input patterns,
generation quality, and controllability. To address these issues, we propose a
novel pipeline called PanoDiff, which efficiently generates complete
360$^\circ$ panoramas using one or more unregistered NFoV images captured from
arbitrary angles. Our approach has two primary components to overcome the
limitations. Firstly, a two-stage angle prediction module to handle various
numbers of NFoV inputs. Secondly, a novel latent diffusion-based panorama
generation model uses incomplete panorama and text prompts as control signals
and utilizes several geometric augmentation schemes to ensure geometric
properties in generated panoramas. Experiments show that PanoDiff achieves
state-of-the-art panoramic generation quality and high controllability, making
it suitable for applications such as content editing.

使用一种称为 PanoDiff 的新型方法，通过使用一个或多个未注册的 Narrow Field-of-View (NFoV) 图像，高效地生成完整的 360° 全景图，该方法通过两个主要组件来克服以往方法中的局限性，包括两阶段的角度预测模块和一种基于潜在扩散的全景生成模型，实验表明 PanoDiff 实现了最先进的全景生成质量和高可控性，适用于内容编辑等应用。

从少量未注册的超广角图像生成 360 度全景

360-Degree Panorama Generation from Few Unregistered NFoV Images

High-quality HDRIs(High Dynamic Range Images), typically HDR panoramas, are
one of the most popular ways to create photorealistic lighting and 360-degree
reflections of 3D scenes in graphics. Given the difficulty of capturing HDRIs,
a versatile and controllable generative model is highly desired, where layman
users can intuitively control the generation process. However, existing
state-of-the-art methods still struggle to synthesize high-quality panoramas
for complex scenes. In this work, we propose a zero-shot text-driven framework,
Text2Light, to generate 4K+ resolution HDRIs without paired training data.
Given a free-form text as the description of the scene, we synthesize the
corresponding HDRI with two dedicated steps: 1) text-driven panorama generation
in low dynamic range(LDR) and low resolution, and 2) super-resolution inverse
tone mapping to scale up the LDR panorama both in resolution and dynamic range.
Specifically, to achieve zero-shot text-driven panorama generation, we first
build dual codebooks as the discrete representation for diverse environmental
textures. Then, driven by the pre-trained CLIP model, a text-conditioned global
sampler learns to sample holistic semantics from the global codebook according
to the input text. Furthermore, a structure-aware local sampler learns to
synthesize LDR panoramas patch-by-patch, guided by holistic semantics. To
achieve super-resolution inverse tone mapping, we derive a continuous
representation of 360-degree imaging from the LDR panorama as a set of
structured latent codes anchored to the sphere. This continuous representation
enables a versatile module to upscale the resolution and dynamic range
simultaneously. Extensive experiments demonstrate the superior capability of
Text2Light in generating high-quality HDR panoramas. In addition, we show the
feasibility of our work in realistic rendering and immersive VR.

本文中提出了一种零样本文本驱动框架 (Text2Light)，它使用自然语言对场景进行描述，并生成适用于复杂场景的 4K + 分辨率 HDR 全景图，其中使用了双重代码本来生成各种环境纹理的离散表示，并由经过预训练的 CLIP 模型驱动的全局采样器和结构感知本地采样器进行指导。同时，提出了一种 360 度图像的连续表示方法，以支持同时提高分辨率和动态范围。

Text2Light: 零样本文本驱动 HDR 全景生成

Text2Light: Zero-Shot Text-Driven HDR Panorama Generation

Image extension models have broad applications in image editing,
computational photography and computer graphics. While image inpainting has
been extensively studied in the literature, it is challenging to directly apply
the state-of-the-art inpainting methods to image extension as they tend to
generate blurry or repetitive pixels with inconsistent semantics. We introduce
semantic conditioning to the discriminator of a generative adversarial network
(GAN), and achieve strong results on image extension with coherent semantics
and visually pleasing colors and textures. We also show promising results in
extreme extensions, such as panorama generation.

本文介绍了一种基于生成对抗网络 (GAN) 的语义条件方法，可在图像扩展、全景图像生成等领域中取得强大的成果。

无穷无尽：用生成对抗网络进行图像扩展

Boundless: Generative Adversarial Networks for Image Extension

Humans can only interact with part of the surrounding environment due to
biological restrictions. Therefore, we learn to reason the spatial
relationships across a series of observations to piece together the surrounding
environment. Inspired by such behavior and the fact that machines also have
computational constraints, we propose \underline{CO}nditional
\underline{CO}ordinate GAN (COCO-GAN) of which the generator generates images
by parts based on their spatial coordinates as the condition. On the other
hand, the discriminator learns to justify realism across multiple assembled
patches by global coherence, local appearance, and edge-crossing continuity.
Despite the full images are never generated during training, we show that
COCO-GAN can produce \textbf{state-of-the-art-quality} full images during
inference. We further demonstrate a variety of novel applications enabled by
teaching the network to be aware of coordinates. First, we perform
extrapolation to the learned coordinate manifold and generate off-the-boundary
patches. Combining with the originally generated full image, COCO-GAN can
produce images that are larger than training samples, which we called
"beyond-boundary generation". We then showcase panorama generation within a
cylindrical coordinate system that inherently preserves horizontally cyclic
topology. On the computation side, COCO-GAN has a built-in divide-and-conquer
paradigm that reduces memory requisition during training and inference,
provides high-parallelism, and can generate parts of images on-demand.

COnditional COordinate GAN (COCO-GAN) 是一种基于空间坐标的图像生成器，通过学习空间关系实现全图像生成，并可实现超出边界和全景生成等多种应用。