This paper introduces FlowMap, an end-to-end differentiable method that
solves for precise camera poses, camera intrinsics, and per-frame dense depth
of a video sequence. Our method performs per-video gradient-descent
minimization of a simple least-squares objective that compares the optical flow
induced by depth, intrinsics, and poses against correspondences obtained via
off-the-shelf optical flow and point tracking. Alongside the use of point
tracks to encourage long-term geometric consistency, we introduce
differentiable re-parameterizations of depth, intrinsics, and pose that are
amenable to first-order optimization. We empirically show that camera
parameters and dense depth recovered by our method enable photo-realistic novel
view synthesis on 360-degree trajectories using Gaussian Splatting. Our method
not only far outperforms prior gradient-descent based bundle adjustment
methods, but surprisingly performs on par with COLMAP, the state-of-the-art SfM
method, on the downstream task of 360-degree novel view synthesis (even though
our method is purely gradient-descent based, fully differentiable, and presents
a complete departure from conventional SfM).

FlowMap 是一种端到端可微分方法，用于解决视频序列的精确相机姿态、相机内参数和帧间密集深度。我们的方法通过基于梯度下降的最小二乘目标对由深度、内参和姿态引起的光流与通过现成的光流和点跟踪获得的对应进行比较来进行视频级梯度下降优化。此外，我们引入了可微分的深度、内参和姿态再参数化方式，以便进行一阶优化。我们的实证结果表明，我们的方法恢复的相机参数和密集深度能够使用高斯赋形在 360 度轨迹上实现逼真的新视角合成。我们的方法不仅在梯度下降的束调整方法方面表现得比之前的方法好得多，而且在 360 度新视角合成的下游任务中，与 COLMAP 这一最先进的 SfM 方法居然表现相当（尽管我们的方法完全基于梯度下降，完全可微分，并且与传统的 SfM 完全不同）。

FlowMap: 高质量相机位姿、内参和深度的梯度下降

FlowMap: High-Quality Camera Poses, Intrinsics, and Depth via Gradient  Descent

Camera rotation estimation from a single image is a challenging task, often
requiring depth data and/or camera intrinsics, which are generally not
available for in-the-wild videos. Although external sensors such as inertial
measurement units (IMUs) can help, they often suffer from drift and are not
applicable in non-inertial reference frames. We present U-ARE-ME, an algorithm
that estimates camera rotation along with uncertainty from uncalibrated RGB
images. Using a Manhattan World assumption, our method leverages the per-pixel
geometric priors encoded in single-image surface normal predictions and
performs optimisation over the SO(3) manifold. Given a sequence of images, we
can use the per-frame rotation estimates and their uncertainty to perform
multi-frame optimisation, achieving robustness and temporal consistency. Our
experiments demonstrate that U-ARE-ME performs comparably to RGB-D methods and
is more robust than sparse feature-based SLAM methods. We encourage the reader
to view the accompanying video at this https URL for
a visual overview of our method.

使用单一 RGB 图像的非标定旋转估计算法 U-ARE-ME，利用曼哈顿世界假设和像素几何先验，在 SO (3) 流形上进行优化，实现对相机旋转及其不确定性的估计，并且通过多帧优化达到稳健性和时间一致性。与 RGB-D 方法相比，U-ARE-ME 性能相当；与稀疏特征为基础的 SLAM 方法相比，U-ARE-ME 更为稳健。可通过附带的视频链接获取我们方法的视觉概述。

U-ARE-ME：曼哈顿环境中的不确定性感知旋转估计

U-ARE-ME: Uncertainty-Aware Rotation Estimation in Manhattan  Environments

We present a novel method for simultaneous learning of depth, egomotion,
object motion, and camera intrinsics from monocular videos, using only
consistency across neighboring video frames as supervision signal. Similarly to
prior work, our method learns by applying differentiable warping to frames and
comparing the result to adjacent ones, but it provides several improvements: We
address occlusions geometrically and differentiably, directly using the depth
maps as predicted during training. We introduce randomized layer normalization,
a novel powerful regularizer, and we account for object motion relative to the
scene. To the best of our knowledge, our work is the first to learn the camera
intrinsic parameters, including lens distortion, from video in an unsupervised
manner, thereby allowing us to extract accurate depth and motion from arbitrary
videos of unknown origin at scale. We evaluate our results on the Cityscapes,
KITTI and EuRoC datasets, establishing new state of the art on depth prediction
and odometry, and demonstrate qualitatively that depth prediction can be
learned from a collection of YouTube videos.

本研究提出了一种新的方法，使用相邻视频帧的一致性作为监督信号，同时从单目视频中学习深度预测、自运动、目标运动和相机内参数，并且首次实现对相机内参进行无监督学习，从而能够在任意未知来源的视频中提取准确的深度和运动信息。此方法在 Cityscapes、KITTI 和 EuRoC 数据集上验证了正确性，并取得了新的最佳性能。