Most existing approaches to autonomous driving fall into one of two
categories: modular pipelines, that build an extensive model of the
environment, and imitation learning approaches, that map images directly to
control outputs. A recently proposed third paradigm, direct perception, aims to
combine the advantages of both by using a neural network to learn appropriate
low-dimensional intermediate representations. However, existing direct
perception approaches are restricted to simple highway situations, lacking the
ability to navigate intersections, stop at traffic lights or respect speed
limits. In this work, we propose a direct perception approach which maps video
input to intermediate representations suitable for autonomous navigation in
complex urban environments given high-level directional inputs. Compared to
state-of-the-art reinforcement and conditional imitation learning approaches,
we achieve an improvement of up to 68 % in goal-directed navigation on the
challenging CARLA simulation benchmark. In addition, our approach is the first
to handle traffic lights and speed signs by using image-level labels only, as
well as smooth car-following, resulting in a significant reduction of traffic
accidents in simulation.

该研究提出了一种基于直接感知的自主驾驶方法，在城市环境中能够根据高级方向输入进行自主导航，同时能够识别交通信号灯和速度标志，降低车辆事故率。

城市环境下驾驶的有条件能力学习

Conditional Affordance Learning for Driving in Urban Environments

The field of functional recognition or affordance estimation from images has
seen a revival in recent years. As originally proposed by Gibson, the
affordances of a scene were directly perceived from the ambient light: in other
words, functional properties like sittable were estimated directly from
incoming pixels. Recent work, however, has taken a mediated approach in which
affordances are derived by first estimating semantics or geometry and then
reasoning about the affordances. In a tribute to Gibson, this paper explores
his theory of affordances as originally proposed. We propose two approaches for
direct perception of affordances and show that they obtain good results and can
out-perform mediated approaches. We hope this paper can rekindle discussion
around direct perception and its implications in the long term.

通过直接感知环境光线，来对场景中的 affordances（如坐着的）属性进行估算，从而避免了先估算语义或几何形状，再推理 affordances 的中介方法，同时提出了两个直接感知 affordances 的方法，并展示它们获得了良好的结果且可以优于中介方法。

捍卫可供性直接知觉

In Defense of the Direct Perception of Affordances

Today, there are two major paradigms for vision-based autonomous driving
systems: mediated perception approaches that parse an entire scene to make a
driving decision, and behavior reflex approaches that directly map an input
image to a driving action by a regressor. In this paper, we propose a third
paradigm: a direct perception approach to estimate the affordance for driving.
We propose to map an input image to a small number of key perception indicators
that directly relate to the affordance of a road/traffic state for driving. Our
representation provides a set of compact yet complete descriptions of the scene
to enable a simple controller to drive autonomously. Falling in between the two
extremes of mediated perception and behavior reflex, we argue that our direct
perception representation provides the right level of abstraction. To
demonstrate this, we train a deep Convolutional Neural Network using recording
from 12 hours of human driving in a video game and show that our model can work
well to drive a car in a very diverse set of virtual environments. We also
train a model for car distance estimation on the KITTI dataset. Results show
that our direct perception approach can generalize well to real driving images.
Source code and data are available on our project website.

本文提出一种基于直接感知的自动驾驶范式，使用深度卷积神经网络将输入图像映射到与驾驶相关的感知指标，并证明该方法可以成功地应用于不同的虚拟环境和现实场景。