Novel view synthesis has observed tremendous developments since the arrival
of NeRFs. However, Nerf models overfit on a single scene, lacking
generalization to out of distribution objects. Recently, diffusion models have
exhibited remarkable performance on introducing generalization in view
synthesis. Inspired by these advancements, we explore the capabilities of a
pretrained stable diffusion model for view synthesis without explicit 3D
priors. Specifically, we base our method on a personalized text to image model,
Dreambooth, given its strong ability to adapt to specific novel objects with a
few shots. Our research reveals two interesting findings. First, we observe
that Dreambooth can learn the high level concept of a view, compared to
arguably more complex strategies which involve finetuning diffusions on large
amounts of multi-view data. Second, we establish that the concept of a view can
be disentangled and transferred to a novel object irrespective of the original
object's identify from which the views are learnt. Motivated by this, we
introduce a learning strategy, FSViewFusion, which inherits a specific view
through only one image sample of a single scene, and transfers the knowledge to
a novel object, learnt from few shots, using low rank adapters. Through
extensive experiments we demonstrate that our method, albeit simple, is
efficient in generating reliable view samples for in the wild images. Code and
models will be released.

通过预训练的稳定扩散模型，基于个性化文本到图像模型 Dreambooth，我们的研究发现视图概念可以被学习和分离，并通过少量示例迁移到新对象，我们引入了一种名为 FSViewFusion 的学习策略，通过单个场景的一张图像样本来传递视图的知识，通过低秩适配器在少量样本上学习新对象，通过大量实验证明我们的方法能够在野外图像中生成可靠的视图样本。