Unveiling an empty street from crowded observations captured by in-car
cameras is crucial for autonomous driving. However, removing all temporary
static objects, such as stopped vehicles and standing pedestrians, presents a
significant challenge. Unlike object-centric 3D inpainting, which relies on
thorough observation in a small scene, street scenes involve long trajectories
that differ from previous 3D inpainting tasks. The camera-centric moving
environment of captured videos further complicates the task due to the limited
degree and time duration of object observation. To address these obstacles, we
introduce StreetUnveiler to reconstruct an empty street. StreetUnveiler learns
a 3D representation of the empty street from crowded observations. Our
representation is based on the hard-label semantic 2D Gaussian Splatting (2DGS)
for its scalability and ability to identify Gaussians to be removed. We inpaint
rendered image after removing unwanted Gaussians to provide pseudo-labels and
subsequently re-optimize the 2DGS. Given its temporal continuous movement, we
divide the empty street scene into observed, partial-observed, and unobserved
regions, which we propose to locate through a rendered alpha map. This
decomposition helps us to minimize the regions that need to be inpainted. To
enhance the temporal consistency of the inpainting, we introduce a novel
time-reversal framework to inpaint frames in reverse order and use later frames
as references for earlier frames to fully utilize the long-trajectory
observations. Our experiments conducted on the street scene dataset
successfully reconstructed a 3D representation of the empty street. The mesh
representation of the empty street can be extracted for further applications.
Project page and more visualizations can be found at:
this https URL

通过 Crowded Observations 捕获的车载相机，我们介绍了 StreetUnveiler 来重建空旷道路的 3D 表示，通过去除不需要的高斯分布，我们使用时间逆转框架对帧进行填充以增强填充的时序一致性。