Modern optical satellite sensors enable high-resolution stereo reconstruction
from space. But the challenging imaging conditions when observing the Earth
from space push stereo matching to its limits. In practice, the resulting
digital surface models (DSMs) are fairly noisy and often do not attain the
accuracy needed for high-resolution applications such as 3D city modeling.
Arguably, stereo correspondence based on low-level image similarity is
insufficient and should be complemented with a-priori knowledge about the
expected surface geometry beyond basic local smoothness. To that end, we
introduce ResDepth, a convolutional neural network that learns such an
expressive geometric prior from example data. ResDepth refines an initial, raw
stereo DSM while conditioning the refinement on the images. I.e., it acts as a
smart, learned post-processing filter and can seamlessly complement any stereo
matching pipeline. In a series of experiments, we find that the proposed method
consistently improves stereo DSMs both quantitatively and qualitatively. We
show that the prior encoded in the network weights captures meaningful
geometric characteristics of urban design, which also generalize across
different districts and even from one city to another. Moreover, we demonstrate
that, by training on a variety of stereo pairs, ResDepth can acquire a
sufficient degree of invariance against variations in imaging conditions and
acquisition geometry.

研究使用卫星传感器进行高分辨率三维城市建模中深度匹配问题，提出了一种基于卷积神经网络的先验学习框架 ResDepth，实现了图像信息和表面几何信息的联合加工，显著提高了数字地面模型的精度。

ResDepth：一种用于高分辨率卫星图像三维重建的深度残差模型

ResDepth: A Deep Residual Prior For 3D Reconstruction From  High-resolution Satellite Images

We propose an embarrassingly simple but very effective scheme for
high-quality dense stereo reconstruction: (i) generate an approximate
reconstruction with your favourite stereo matcher; (ii) rewarp the input images
with that approximate model; (iii) with the initial reconstruction and the
warped images as input, train a deep network to enhance the reconstruction by
regressing a residual correction; and (iv) if desired, iterate the refinement
with the new, improved reconstruction. The strategy to only learn the residual
greatly simplifies the learning problem. A standard Unet without bells and
whistles is enough to reconstruct even small surface details, like dormers and
roof substructures in satellite images. We also investigate residual
reconstruction with less information and find that even a single image is
enough to greatly improve an approximate reconstruction. Our full model reduces
the mean absolute error of state-of-the-art stereo reconstruction systems by
>50%, both in our target domain of satellite stereo and on stereo pairs from
the ETH3D benchmark.

使用深度学习的残差学习方法来提高立体重建质量，并可用于卫星图像等领域。在 ETH3D 基准和卫星图像领域，平均绝对误差降低了 50% 以上。

ResDepth: 学习残差三维重建

ResDepth: Learned Residual Stereo Reconstruction

In order to facilitate further research in stereo reconstruction with
multi-date satellite images, the goal of this paper is to provide a set of
stereo-rectified images and the associated groundtruthed disparities for 10
AOIs (Area of Interest) drawn from two sources: 8 AOIs from IARPA's MVS
Challenge dataset and 2 AOIs from the CORE3D-Public dataset. The disparities
were groundtruthed by first constructing a fused DSM from the stereo pairs and
by aligning 30 cm LiDAR with the fused DSM. Unlike the existing benckmarking
datasets, we have also carried out a quantitative evaluation of our
groundtruthed disparities using human annotated points in two of the AOIs.
Additionally, the rectification accuracy in our dataset is comparable to the
same in the existing state-of-the-art stereo datasets. In general, we have used
the WorldView-3 (WV3) images for the dataset, the exception being the UCSD area
for which we have used both WV3 and WorldView-2 (WV2) images. All of the
dataset images are now in the public domain. Since multi-date satellite images
frequently include images acquired in different seasons (which creates
challenges in finding corresponding pairs of pixels for stereo), our dataset
also includes for each image a building mask over which the disparities
estimated by stereo should prove reliable. Additional metadata included in the
dataset includes information about each image's acquisition date and time, the
azimuth and elevation angles of the camera, and the intersection angles for the
two views in a stereo pair. Also included in the dataset are both quantitative
and qualitative analyses of the accuracy of the groundtruthed disparity maps.
Our dataset is available for download at
https://engineering.purdue.edu/RVL/Database/SatStereo/index.html

本文提供了一组经过立体校正的卫星图像及其对应的地面真实位移图，以供研究者进行立体重建的进一步研究，该数据集包括来自两个源的 10 个感兴趣区域，其中 8 个来自 IARPA 的 MVS Challenge 数据集，2 个来自 CORE3D-Public 数据集。

卫星图像新的立体基准数据集

A New Stereo Benchmarking Dataset for Satellite Images

The matching function for the problem of stereo reconstruction or optical
flow has been traditionally designed as a function of the distance between the
features describing matched pixels. This approach works under assumption, that
the appearance of pixels in two stereo cameras or in two consecutive video
frames does not change dramatically. However, this might not be the case, if we
try to match pixels over a large interval of time.
In this paper we propose a method, which learns the matching function, that
automatically finds the space of allowed changes in visual appearance, such as
due to the motion blur, chromatic distortions, different colour calibration or
seasonal changes. Furthermore, it automatically learns the importance of
matching scores of contextual features at different relative locations and
scales. Proposed classifier gives reliable estimations of pixel disparities
already without any form of regularization.
We evaluated our method on two standard problems - stereo matching on KITTI
outdoor dataset, optical flow on Sintel data set, and on newly introduced
TimeLapse change detection dataset. Our algorithm obtained very promising
results comparable to the state-of-the-art.

本篇论文介绍了一种学习匹配函数的方法，该方法能自动发现允许视觉外观变化的空间，并学习相关特征在不同相对位置和尺度上的匹配分数的重要性。该算法已在 KITTI、Sintel 和 TimeLapse 数据集上实现了有希望的结果。