Learning single image depth estimation model from monocular video sequence is
a very challenging problem. In this paper, we propose a novel training loss
which enables us to include more images for supervision during the training
process. We propose a simple yet effective model to account the frame to frame
pixel motion. We also design a novel network architecture for single image
estimation. When combined, our method produces state of the art results for
monocular depth estimation on the KITTI dataset in the self-supervised setting.

从单目视频序列中学习单张图像深度估计模型是一个非常具有挑战性的问题。在本文中，我们提出了一种新颖的训练损失，使我们能够在训练过程中包含更多图像以进行监督。我们提出了一个简单而有效的模型来考虑帧与帧之间的像素运动。同时，我们还设计了一种新颖的网络架构来进行单张图像估计。当结合在一起时，我们的方法在自监督设置下，在 KITTI 数据集上产生了最先进的结果。

从单目视频序列中学习深度

Learning depth from monocular video sequences

Depth estimation, visual odometry (VO), and bird's-eye-view (BEV) scene
layout estimation present three critical tasks for driving scene perception,
which is fundamental for motion planning and navigation in autonomous driving.
Though they are complementary to each other, prior works usually focus on each
individual task and rarely deal with all three tasks together. A naive way is
to accomplish them independently in a sequential or parallel manner, but there
are many drawbacks, i.e., 1) the depth and VO results suffer from the inherent
scale ambiguity issue; 2) the BEV layout is directly predicted from the
front-view image without using any depth-related information, although the
depth map contains useful geometry clues for inferring scene layouts. In this
paper, we address these issues by proposing a novel joint perception framework
named JPerceiver, which can simultaneously estimate scale-aware depth and VO as
well as BEV layout from a monocular video sequence. It exploits the cross-view
geometric transformation (CGT) to propagate the absolute scale from the road
layout to depth and VO based on a carefully-designed scale loss. Meanwhile, a
cross-view and cross-modal transfer (CCT) module is devised to leverage the
depth clues for reasoning road and vehicle layout through an attention
mechanism. JPerceiver can be trained in an end-to-end multi-task learning way,
where the CGT scale loss and CCT module promote inter-task knowledge transfer
to benefit feature learning of each task. Experiments on Argoverse, Nuscenes
and KITTI show the superiority of JPerceiver over existing methods on all the
above three tasks in terms of accuracy, model size, and inference speed. The
code and models are available
at~\href{https://github.com/sunnyHelen/JPerceiver}{this https URL}.

该研究提出了一种名为 JPerceiver 的新方法，通过多任务学习的方式，利用 CGT 比例损失和 CCT 模块促进跨任务知识转移，从单目视频序列中同时估计深度、VO 和 BEV 布局，是当前三项任务中准确性、模型尺寸和推理速度方面最优秀的方法。

JPerceiver: 自动驾驶全感知网络

JPerceiver: Joint Perception Network for Depth, Pose and Layout  Estimation in Driving Scenes

We introduce the problem of perpetual view generation - long-range generation
of novel views corresponding to an arbitrarily long camera trajectory given a
single image. This is a challenging problem that goes far beyond the
capabilities of current view synthesis methods, which quickly degenerate when
presented with large camera motions. Methods for video generation also have
limited ability to produce long sequences and are often agnostic to scene
geometry. We take a hybrid approach that integrates both geometry and image
synthesis in an iterative `\emph{render}, \emph{refine} and \emph{repeat}'
framework, allowing for long-range generation that cover large distances after
hundreds of frames. Our approach can be trained from a set of monocular video
sequences. We propose a dataset of aerial footage of coastal scenes, and
compare our method with recent view synthesis and conditional video generation
baselines, showing that it can generate plausible scenes for much longer time
horizons over large camera trajectories compared to existing methods. Project
page at this https URL

介绍了一个基于迭代式渲染、调整和重复的框架的方法，通过一个单一的图像生成相应的漫长的视角图，该方法能够比现有的方法更好地生成海岸场景的长时间视角图。