In this paper, we propose an algorithm that allows joint refinement of camera
pose and scene geometry represented by decomposed low-rank tensor, using only
2D images as supervision. First, we conduct a pilot study based on a 1D signal
and relate our findings to 3D scenarios, where the naive joint pose
optimization on voxel-based NeRFs can easily lead to sub-optimal solutions.
Moreover, based on the analysis of the frequency spectrum, we propose to apply
convolutional Gaussian filters on 2D and 3D radiance fields for a
coarse-to-fine training schedule that enables joint camera pose optimization.
Leveraging the decomposition property in decomposed low-rank tensor, our method
achieves an equivalent effect to brute-force 3D convolution with only incurring
little computational overhead. To further improve the robustness and stability
of joint optimization, we also propose techniques of smoothed 2D supervision,
randomly scaled kernel parameters, and edge-guided loss mask. Extensive
quantitative and qualitative evaluations demonstrate that our proposed
framework achieves superior performance in novel view synthesis as well as
rapid convergence for optimization.

我们提出了一个算法，允许使用仅具有 2D 图像作为监督来共同优化由分解低秩张量表示的相机姿态和场景几何，利用分解低秩张量的分解属性，我们的方法在仅产生少量计算开销的情况下实现了等效于蛮力 3D 卷积的效果。