High resolution panoramic video content is paramount for immersive experiences in Virtual Reality, but is non-trivial to collect as it requires specialized equipment and intricate camera setups. In this work, we introduce VideoPanda, a novel approach for synthesizing 360$^\circ$ videos conditioned on text or single-view video data. VideoPanda leverages multi-view attention layers to augment a video diffusion model, enabling it to generate consistent multi-view videos that can be combined into immersive panoramic content. VideoPanda is trained jointly using two conditions: text-only and single-view video, and supports autoregressive generation of long-videos. To overcome the computational burden of multi-view video generation, we randomly subsample the duration and camera views used during training and show that the model is able to gracefully generalize to generating more frames during inference. Extensive evaluations on both real-world and synthetic video datasets demonstrate that VideoPanda generates more realistic and coherent 360$^\circ$ panoramas across all input conditions compared to existing methods. Visit the project website at https://research-staging.nvidia.com/labs/toronto-ai/VideoPanda/ for results.

本研究解决了在虚拟现实中生成高分辨率全景视频内容的挑战，尤其是其收集过程复杂且需要专业设备。论文提出了一种新颖的VideoPanda方法，通过多视角注意力层增强视频扩散模型，实现了基于文本或单视角视频数据生成一致的多视角视频，并在生成更长视频时表现出良好的泛化能力。实验证明，与现有方法相比，VideoPanda在现实和合成视频数据集上生成的360°全景图像更为真实和连贯。

VideoPanda：基于多视角注意力的视频全景扩散