Neural radiance fields (NeRF) methods have demonstrated impressive novel view
synthesis performance. The core approach is to render individual rays by
querying a neural network at points sampled along the ray to obtain the density
and colour of the sampled points, and integrating this information using the
rendering equation. Since dense sampling is computationally prohibitive, a
common solution is to perform coarse-to-fine sampling.
In this work we address a clear limitation of the vanilla coarse-to-fine
approach -- that it is based on a heuristic and not trained end-to-end for the
task at hand. We introduce a differentiable module that learns to propose
samples and their importance for the fine network, and consider and compare
multiple alternatives for its neural architecture. Training the proposal module
from scratch can be unstable due to lack of supervision, so an effective
pre-training strategy is also put forward. The approach, named `NeRF in detail'
(NeRF-ID), achieves superior view synthesis quality over NeRF and the
state-of-the-art on the synthetic Blender benchmark and on par or better
performance on the real LLFF-NeRF scenes. Furthermore, by leveraging the
predicted sample importance, a 25% saving in computation can be achieved
without significantly sacrificing the rendering quality.

作者提出了基于可微分不可视模块的 NeRF-ID 方法，该方法可以优化粗到精的采样策略，得到比 NeRF 和当前最先进算法更好的视图合成质量，并且显著降低计算时间。

NeRF 详解：学习采样进行视图合成

NeRF in detail: Learning to sample for view synthesis

The panoptic segmentation task requires a unified result from semantic and
instance segmentation outputs that may contain overlaps. However, current
studies widely ignore modeling overlaps. In this study, we aim to model overlap
relations among instances and resolve them for panoptic segmentation. Inspired
by scene graph representation, we formulate the overlapping problem as a
simplified case, named scene overlap graph. We leverage each object's category,
geometry and appearance features to perform relational embedding, and output a
relation matrix that encodes overlap relations. In order to overcome the lack
of supervision, we introduce a differentiable module to resolve the overlap
between any pair of instances. The mask logits after removing overlaps are fed
into per-pixel instance \verb|id| classification, which leverages the panoptic
supervision to assist in the modeling of overlap relations. Besides, we
generate an approximate ground truth of overlap relations as the weak
supervision, to quantify the accuracy of overlap relations predicted by our
method. Experiments on COCO and Cityscapes demonstrate that our method is able
to accurately predict overlap relations, and outperform the state-of-the-art
performance for panoptic segmentation. Our method also won the Innovation Award
in COCO 2019 challenge.

本研究旨在建立一种基于场景图表示的简化情形，名为场景重合图，来描述实体之间的重叠关系，并利用关系嵌入方法，以减少监督来解决实例之间的重叠关系。最终的试验结果表明，我们提出的方法能够准确地预测重叠关系，在所有测试集上的表现均优于现有最先进的方法，在 COCO 2019 基准测试中获得了创新奖。