Accurate perception of the dynamic environment is a fundamental task for autonomous driving and robot systems. This paper introduces Let Occ Flow, the first self-supervised work for joint 3D occupancy and occupancy flow prediction using only camera inputs, eliminating the need for 3D annotations. Utilizing TPV for unified scene representation and deformable attention layers for feature aggregation, our approach incorporates a backward-forward temporal attention module to capture dynamic object dependencies, followed by a 3D refine module for fine-gained volumetric representation. Besides, our method extends differentiable rendering to 3D volumetric flow fields, leveraging zero-shot 2D segmentation and optical flow cues for dynamic decomposition and motion optimization. Extensive experiments on nuScenes and KITTI datasets demonstrate the competitive performance of our approach over prior state-of-the-art methods.

这篇论文介绍了Let Occ Flow，它是第一个仅使用相机输入进行联合3D占用和占用流预测的自监督工作，消除了对3D注释的需求。利用统一场景表示的TPV和可变形注意力层进行特征聚合，我们的方法采用前后时间注意模块捕捉动态物体之间的依赖关系，并使用3D精细模块进行体积表示。此外，我们的方法将可微分渲染扩展到3D体积流场，利用零样本2D分割和光流线索进行动态分解和运动优化。在nuScenes和KITTI数据集上进行的大量实验证明了我们方法在之前最先进方法上的竞争性表现。

Let Occ Flow: 自监督式 3D 占据流预测