Street-view image has been widely applied as a crucial mobile mapping data
source. The inpainting of street-view images is a critical step for street-view
image processing, not only for the privacy protection, but also for the urban
environment mapping applications. This paper presents a novel Deep Neural
Network (DNN), multi-scale semantic prior Feature guided image inpainting
Network (MFN) for inpainting street-view images, which generate static
street-view images without moving objects (e.g., pedestrians, vehicles). To
enhance global context understanding, a semantic prior prompter is introduced
to learn rich semantic priors from large pre-trained model. We design the
prompter by stacking multiple Semantic Pyramid Aggregation (SPA) modules,
capturing a broad range of visual feature patterns. A semantic-enhanced image
generator with a decoder is proposed that incorporates a novel cascaded
Learnable Prior Transferring (LPT) module at each scale level. For each decoder
block, an attention transfer mechanism is applied to capture long-term
dependencies, and the semantic prior features are fused with the image features
to restore plausible structure in an adaptive manner. Additionally, a
background-aware data processing scheme is adopted to prevent the generation of
hallucinated objects within holes. Experiments on Apolloscapes and Cityscapes
datasets demonstrate better performance than state-of-the-art methods, with
MAE, and LPIPS showing improvements of about 9.5% and 41.07% respectively.
Visual comparison survey among multi-group person is also conducted to provide
performance evaluation, and the results suggest that the proposed MFN offers a
promising solution for privacy protection and generate more reliable scene for
urban applications with street-view images.

这篇论文介绍了一种深度神经网络（DNN），多尺度语义先验特征引导图像修复网络（MFN），用于修补街景图像，以实现隐私保护并为城市应用生成可靠场景。

用于城市街景图像的多尺度语义先验特征引导深度神经网络

Multi-scale Semantic Prior Features Guided Deep Neural Network for Urban  Street-view Image

To find the geolocation of a street-view image, cross-view geolocalization
(CVGL) methods typically perform image retrieval on a database of georeferenced
aerial images and determine the location from the visually most similar match.
Recent approaches focus mainly on settings where street-view and aerial images
are preselected to align w.r.t. translation or orientation, but struggle in
challenging real-world scenarios where varying camera poses have to be matched
to the same aerial image. We propose a novel trainable retrieval architecture
that uses bird's eye view (BEV) maps rather than vectors as embedding
representation, and explicitly addresses the many-to-one ambiguity that arises
in real-world scenarios. The BEV-based retrieval is trained using the same
contrastive setting and loss as classical retrieval.
Our method C-BEV surpasses the state-of-the-art on the retrieval task on
multiple datasets by a large margin. It is particularly effective in
challenging many-to-one scenarios, e.g. increasing the top-1 recall on VIGOR's
cross-area split with unknown orientation from 31.1% to 65.0%. Although the
model is supervised only through a contrastive objective applied on image
pairings, it additionally learns to infer the 3-DoF camera pose on the matching
aerial image, and even yields a lower mean pose error than recent methods that
are explicitly trained with metric groundtruth.

使用鸟瞰图作为嵌入表示的新型可训练的检索体系结构，在交叉视图地理定位任务中表现出色，特别在挑战性的多对一情景中有效，并且能够推断匹配航空图像上的 3 自由度摄像机姿态，甚至比明确通过度量真值进行培训的最新方法具有更低的平均姿态误差。