Efficient processing of high-res video streams is safety-critical for many
robotics applications such as autonomous driving. To maintain real-time
performance, many practical systems downsample the video stream. But this can
hurt downstream tasks such as (small) object detection. Instead, we take
inspiration from biological vision systems that allocate more foveal "pixels"
to salient parts of the scene. We introduce FOVEA, an approach for intelligent
downsampling that ensures salient image regions remain "magnified" in the
downsampled output. Given a high-res image, FOVEA applies a differentiable
resampling layer that outputs a small fixed-size image canvas, which is then
processed with a differentiable vision module (e.g., object detection network),
whose output is then differentiably backward mapped onto the original image
size. The key idea is to resample such that background pixels can make room for
salient pixels of interest. In order to ensure the overall pipeline remains
efficient, FOVEA makes use of cheap and readily available cues for saliency,
including dataset-specific spatial priors or temporal priors computed from
object predictions in the recent past. On the autonomous driving datasets
Argoverse-HD and BDD100K, our proposed method boosts the detection AP over
standard Faster R-CNN, both with and without finetuning. Without any noticeable
increase in compute, we improve accuracy on small objects by over 2x without
degrading performance on large objects. Finally, FOVEA sets a new record for
streaming AP (from 17.8 to 23.0 on a GTX 1080 Ti GPU), a metric designed to
capture both accuracy and latency.

FOVEA 是一种基于人眼视觉特点，通过可微分的重采样层对高分辨率图像进行智能降采样，保证感兴趣区域的像素不会丢失，在自动驾驶数据集 Argoverse-HD 和 BDD100K 上，能够提升小物体检测的准确率而不降低大物体的准确率，并创造了流媒体 AP（同时考虑准确性和延迟）的新记录。

FOVEA: 自主导航用凹坑视觉图像放大技术

FOVEA: Foveated Image Magnification for Autonomous Navigation

Psychlab is a simulated psychology laboratory inside the first-person 3D game
world of DeepMind Lab (Beattie et al. 2016). Psychlab enables implementations
of classical laboratory psychological experiments so that they work with both
human and artificial agents. Psychlab has a simple and flexible API that
enables users to easily create their own tasks. As examples, we are releasing
Psychlab implementations of several classical experimental paradigms including
visual search, change detection, random dot motion discrimination, and multiple
object tracking. We also contribute a study of the visual psychophysics of a
specific state-of-the-art deep reinforcement learning agent: UNREAL (Jaderberg
et al. 2016). This study leads to the surprising conclusion that UNREAL learns
more quickly about larger target stimuli than it does about smaller stimuli. In
turn, this insight motivates a specific improvement in the form of a simple
model of foveal vision that turns out to significantly boost UNREAL's
performance, both on Psychlab tasks, and on standard DeepMind Lab tasks. By
open-sourcing Psychlab we hope to facilitate a range of future such studies
that simultaneously advance deep reinforcement learning and improve its links
with cognitive science.

Psychlab 是 DeepMind Lab 中的模拟心理学实验室，可用于实现经典实验范式和深度强化学习代理的视觉心理物理研究。实验结果表明，UNREAL 的学习速度受目标刺激大小的影响，并通过提供简单的视觉准则来改善 UNREAL 在任务中的表现。

Psychlab：用于深度强化学习代理的心理学实验室

Psychlab: A Psychology Laboratory for Deep Reinforcement Learning Agents

Foveal vision makes up less than 1% of the visual field. The other 99% is
peripheral vision. Precisely what human beings see in the periphery is both
obvious and mysterious in that we see it with our own eyes but can't visualize
what we see, except in controlled lab experiments. Degradation of information
in the periphery is far more complex than what might be mimicked with a radial
blur. Rather, behaviorally-validated models hypothesize that peripheral vision
measures a large number of local texture statistics in pooling regions that
overlap and grow with eccentricity. In this work, we develop a new method for
peripheral vision simulation by training a generative neural network on a
behaviorally-validated full-field synthesis model. By achieving a 21,000 fold
reduction in running time, our approach is the first to combine realism and
speed of peripheral vision simulation to a degree that provides a whole new way
to approach visual design: through peripheral visualization.

本文提出了一种基于生成神经网络的外围视觉模拟方法，该方法能够实现对局部纹理统计学的快速测量，并为视觉设计提供一种全新的方法。