We present DietNeRF, a 3D neural scene representation estimated from a few
images. Neural Radiance Fields (NeRF) learn a continuous volumetric
representation of a scene through multi-view consistency, and can be rendered
from novel viewpoints by ray casting. While NeRF has an impressive ability to
reconstruct geometry and fine details given many images, up to 100 for
challenging 360{\deg} scenes, it often finds a degenerate solution to its image
reconstruction objective when only a few input views are available. To improve
few-shot quality, we propose DietNeRF. We introduce an auxiliary semantic
consistency loss that encourages realistic renderings at novel poses. DietNeRF
is trained on individual scenes to (1) correctly render given input views from
the same pose, and (2) match high-level semantic attributes across different,
random poses. Our semantic loss allows us to supervise DietNeRF from arbitrary
poses. We extract these semantics using a pre-trained visual encoder such as
CLIP, a Vision Transformer trained on hundreds of millions of diverse
single-view, 2D photographs mined from the web with natural language
supervision. In experiments, DietNeRF improves the perceptual quality of
few-shot view synthesis when learned from scratch, can render novel views with
as few as one observed image when pre-trained on a multi-view dataset, and
produces plausible completions of completely unobserved regions.

DietNeRF 通过引入辅助语义一致性损失，使用预训练的视觉编码器，改进了使用较少图像时的神经辐射场的视图合成，从而提高了感知质量。