We introduce HyperFields, a method for generating text-conditioned Neural
Radiance Fields (NeRFs) with a single forward pass and (optionally) some
fine-tuning. Key to our approach are: (i) a dynamic hypernetwork, which learns
a smooth mapping from text token embeddings to the space of NeRFs; (ii) NeRF
distillation training, which distills scenes encoded in individual NeRFs into
one dynamic hypernetwork. These techniques enable a single network to fit over
a hundred unique scenes. We further demonstrate that HyperFields learns a more
general map between text and NeRFs, and consequently is capable of predicting
novel in-distribution and out-of-distribution scenes -- either zero-shot or
with a few finetuning steps. Finetuning HyperFields benefits from accelerated
convergence thanks to the learned general map, and is capable of synthesizing
novel scenes 5 to 10 times faster than existing neural optimization-based
methods. Our ablation experiments show that both the dynamic architecture and
NeRF distillation are critical to the expressivity of HyperFields.

我们介绍了 HyperFields，一种能够在单次前向传递和（可选地）某些微调中生成以文本为条件的神经辐射场（NeRFs）的方法。我们方法的关键在于：（i）动态超网络，能够将文本标记嵌入与 NeRFs 空间之间的映射平滑化学习；（ii）NeRF 蒸馏训练，将编码在各个 NeRFs 中的场景蒸馏成一个动态超网络。这些技术使得单个网络可以适应超过一百个独特场景。我们进一步证明 HyperFields 学习了文本和 NeRFs 之间更通用的映射，因此能够预测新的分布内和分布外场景 —— 零样本或经过少量微调步骤。基于学习到的通用映射，对 HyperFields 进行微调能够加速收敛，并且能够比现有的基于神经优化的方法更快地合成新场景（速度提高了 5 到 10 倍）。我们的消融实验表明，动态架构和 NeRF 蒸馏对于 HyperFields 的表达能力至关重要。

超场：从文本中实现零样本生成 NeRF

HyperFields: Towards Zero-Shot Generation of NeRFs from Text

Leveraging large-scale image-text datasets and advancements in diffusion
models, text-driven generative models have made remarkable strides in the field
of image generation and editing. This study explores the potential of extending
the text-driven ability to the generation and editing of multi-text conditioned
long videos. Current methodologies for video generation and editing, while
innovative, are often confined to extremely short videos (typically less than
24 frames) and are limited to a single text condition. These constraints
significantly limit their applications given that real-world videos usually
consist of multiple segments, each bearing different semantic information. To
address this challenge, we introduce a novel paradigm dubbed as Gen-L-Video,
capable of extending off-the-shelf short video diffusion models for generating
and editing videos comprising hundreds of frames with diverse semantic segments
without introducing additional training, all while preserving content
consistency. We have implemented three mainstream text-driven video generation
and editing methodologies and extended them to accommodate longer videos imbued
with a variety of semantic segments with our proposed paradigm. Our
experimental outcomes reveal that our approach significantly broadens the
generative and editing capabilities of video diffusion models, offering new
possibilities for future research and applications. The code is available at
this https URL

本文提出了一种称为 Gen-L-Video 的新方法，该方法利用现有的短视频扩散模型扩展了当前的文本驱动视频生成和编辑能力，使其能够生成和编辑具有多个语义段的数百帧的长视频，而不需要额外的训练，从而极大地拓宽了视频扩散模型的生成和编辑能力。