Precise scene understanding is key for most robot monitoring and intervention
tasks in agriculture. In this work we present PAg-NeRF which is a novel
NeRF-based system that enables 3D panoptic scene understanding. Our
representation is trained using an image sequence with noisy robot odometry
poses and automatic panoptic predictions with inconsistent IDs between frames.
Despite this noisy input, our system is able to output scene geometry,
photo-realistic renders and 3D consistent panoptic representations with
consistent instance IDs. We evaluate this novel system in a very challenging
horticultural scenario and in doing so demonstrate an end-to-end trainable
system that can make use of noisy robot poses rather than precise poses that
have to be pre-calculated. Compared to a baseline approach the peak signal to
noise ratio is improved from 21.34dB to 23.37dB while the panoptic quality
improves from 56.65% to 70.08%. Furthermore, our approach is faster and can be
tuned to improve inference time by more than a factor of 2 while being memory
efficient with approximately 12 times fewer parameters.

使用 PAg-NeRF 这一新颖的基于 NeRF 的系统，我们能够在具有嘈杂机器人测距位姿和不一致的自动全景预测 ID 之间的图像序列上训练我们的表示，尽管有这些嘈杂的输入，我们的系统能够输出场景几何结构，照片逼真渲染和一致的 3D 全景表示，我们在一个极具挑战的园艺场景中评估了这一新颖系统，通过这样做证明了一个可以利用嘈杂机器人位姿而不是预先计算的精确位姿的端到端可训练系统，与基准方法相比，峰值信噪比从 21.34dB 提高到 23.37dB，全景质量从 56.65% 提高到 70.08%，此外，我们的方法更快且可以通过调整提高推理时间，同时具有大约 12 倍少的参数。