We introduce Free3D, a simple approach designed for open-set novel view
synthesis (NVS) from a single image. Similar to Zero-1-to-3, we start from a
pre-trained 2D image generator for generalization, and fine-tune it for NVS.
Compared to recent and concurrent works, we obtain significant improvements
without resorting to an explicit 3D representation, which is slow and
memory-consuming or training an additional 3D network. We do so by encoding
better the target camera pose via a new per-pixel ray conditioning
normalization (RCN) layer. The latter injects pose information in the
underlying 2D image generator by telling each pixel its specific viewing
direction. We also improve multi-view consistency via a light-weight multi-view
attention layer and multi-view noise sharing. We train Free3D on the Objaverse
dataset and demonstrate excellent generalization to various new categories in
several new datasets, including OminiObject3D and GSO. We hope our simple and
effective approach will serve as a solid baseline and help future research in
NVS with more accuracy pose. The project page is available at
this https URL

这篇研究提出了一种名为 Free3D 的简单方法，用于通过单张图像进行开放集合的新视角合成。使用编码目标相机姿势的新的像素级射线调节归一化（RCN）层，改进了目标相机姿势的建模。同时通过轻量级多视图注意力层和多视图噪声共享提高了多视图一致性。我们在 Objaverse 数据集上训练了 Free3D，并在几个新数据集中展示了其卓越的泛化性能。希望我们的简单而有效的方法能够作为一个坚实的基准，并有助于未来关于新视角合成更精确的研究。