Large image diffusion models have demonstrated zero-shot capability in novel
view synthesis (NVS). However, existing diffusion-based NVS methods struggle to
generate novel views that are accurately consistent with the corresponding
ground truth poses and appearances, even on the training set. This consequently
limits the performance of downstream tasks, such as image-to-multiview
generation and 3D reconstruction. We realize that such inconsistency is largely
due to the fact that it is difficult to enforce accurate pose and appearance
alignment directly in the diffusion training, as mostly done by existing
methods such as Zero123. To remedy this problem, we propose Ctrl123, a
closed-loop transcription-based NVS diffusion method that enforces alignment
between the generated view and ground truth in a pose-sensitive feature space.
Our extensive experiments demonstrate the effectiveness of Ctrl123 on the tasks
of NVS and 3D reconstruction, achieving significant improvements in both
multiview-consistency and pose-consistency over existing methods.

Ctrl123 是一种基于封闭循环转录的新颖视图合成扩散方法，它在姿态敏感特征空间中强制生成视角和真实数据的一致性，在 NVS 和 3D 重建任务中取得了显著的改进。