It is well known that semantic segmentation can be used as an effective
intermediate representation for learning driving policies. However, the task of
street scene semantic segmentation requires expensive annotations. Furthermore,
segmentation algorithms are often trained irrespective of the actual driving
task, using auxiliary image-space loss functions which are not guaranteed to
maximize driving metrics such as safety or distance traveled per intervention.
In this work, we seek to quantify the impact of reducing segmentation
annotation costs on learned behavior cloning agents. We analyze several
segmentation-based intermediate representations. We use these visual
abstractions to systematically study the trade-off between annotation
efficiency and driving performance, i.e., the types of classes labeled, the
number of image samples used to learn the visual abstraction model, and their
granularity (e.g., object masks vs. 2D bounding boxes). Our analysis uncovers
several practical insights into how segmentation-based visual abstractions can
be exploited in a more label efficient manner. Surprisingly, we find that
state-of-the-art driving performance can be achieved with orders of magnitude
reduction in annotation cost. Beyond label efficiency, we find several
additional training benefits when leveraging visual abstractions, such as a
significant reduction in the variance of the learned policy when compared to
state-of-the-art end-to-end driving models.

研究降低语义分割标注成本对行为克隆代理学习的影响，发现利用视觉抽象表示可实现更佳的标注效率和驾驶性能，并发现了其他训练优势。