An environment representation (ER) is a substantial part of every autonomous
system. It introduces a common interface between perception and other system
components, such as decision making, and allows downstream algorithms to deal
with abstracted data without knowledge of the used sensor. In this work, we
propose and evaluate a novel architecture that generates an egocentric,
grid-based, predictive, and semantically-interpretable ER. In particular, we
provide a proof of concept for the spatio-temporal fusion of multiple camera
sequences and short-term prediction in such an ER. Our design utilizes a strong
semantic segmentation network together with depth and egomotion estimates to
first extract semantic information from multiple camera streams and then
transform these separately into egocentric temporally-aligned bird's-eye view
grids. A deep encoder-decoder network is trained to fuse a stack of these grids
into a unified semantic grid representation and to predict the dynamics of its
surrounding. We evaluate this representation on real-world sequences of the
Cityscapes dataset and show that our architecture can make accurate predictions
in complex sensor fusion scenarios and significantly outperforms a model-driven
baseline in a category-based evaluation.

本文提出并评估了一种新型架构，该架构生成一种自我中心的，基于网格的，预测性和语义可解释的环境表示。通过多个相机流的空间 - 时间融合，深度编码器 - 解码器网络被训练来将这些视图融合为一个统一的语义网格表示，并预测其周围环境的动态。