Imaging systems consist of cameras to encode visual information about the
world and perception models to interpret this encoding. Cameras contain (1)
illumination sources, (2) optical elements, and (3) sensors, while perception
models use (4) algorithms. Directly searching over all combinations of these
four building blocks to design an imaging system is challenging due to the size
of the search space. Moreover, cameras and perception models are often designed
independently, leading to sub-optimal task performance. In this paper, we
formulate these four building blocks of imaging systems as a context-free
grammar (CFG), which can be automatically searched over with a learned camera
designer to jointly optimize the imaging system with task-specific perception
models. By transforming the CFG to a state-action space, we then show how the
camera designer can be implemented with reinforcement learning to intelligently
search over the combinatorial space of possible imaging system configurations.
We demonstrate our approach on two tasks, depth estimation and camera rig
design for autonomous vehicles, showing that our method yields rigs that
outperform industry-wide standards. We believe that our proposed approach is an
important step towards automating imaging system design.

通过将成像系统的四个组成部分组织成上下文无关语法（CFG），并通过学习的相机设计器，以任务特定的感知模型共同优化成像系统，我们展示了一种通过强化学习实现相机设计师的智能搜索策略，用于智能搜索可能成像系统配置的组合空间，以实现两个任务：深度估计和自动驾驶车辆相机架设计。结果表明，我们的方法优于行业标准，并且认为我们的方法是自动化成像系统设计的重要一步。