Generating multiview images from a single view facilitates the rapid
generation of a 3D mesh conditioned on a single image. Recent methods that
introduce 3D global representation into diffusion models have shown the
potential to generate consistent multiviews, but they have reduced generation
speed and face challenges in maintaining generalizability and quality. To
address this issue, we propose EpiDiff, a localized interactive multiview
diffusion model. At the core of the proposed approach is to insert a
lightweight epipolar attention block into the frozen diffusion model,
leveraging epipolar constraints to enable cross-view interaction among feature
maps of neighboring views. The newly initialized 3D modeling module preserves
the original feature distribution of the diffusion model, exhibiting
compatibility with a variety of base diffusion models. Experiments show that
EpiDiff generates 16 multiview images in just 12 seconds, and it surpasses
previous methods in quality evaluation metrics, including PSNR, SSIM and LPIPS.
Additionally, EpiDiff can generate a more diverse distribution of views,
improving the reconstruction quality from generated multiviews. Please see our
project page at this https URL

通过将轻量级的极线注意力块插入冻结扩散模型，EpiDiff 提出了一种局部交互多视角扩散模型，实现了对单一图像条件下的三维网格的快速生成，并在生成速度、泛化能力和质量方面具有优势。