The performance of robots in their applications heavily depends on the
quality of sensory input. However, designing sensor payloads and their
parameters for specific robotic tasks is an expensive process that requires
well-established sensor knowledge and extensive experiments with physical
hardware. With cameras playing a pivotal role in robotic perception, we
introduce a novel end-to-end optimization approach for co-designing a camera
with specific robotic tasks by combining derivative-free and gradient-based
optimizers. The proposed method leverages recent computer graphics techniques
and physical camera characteristics to prototype the camera in software,
simulate operational environments and tasks for robots, and optimize the camera
design based on the desired tasks in a cost-effective way. We validate the
accuracy of our camera simulation by comparing it with physical cameras, and
demonstrate the design of cameras with stronger performance than common
off-the-shelf alternatives. Our approach supports the optimization of both
continuous and discrete camera parameters, manufacturing constraints, and can
be generalized to a broad range of camera design scenarios including multiple
cameras and unconventional cameras. This work advances the fully automated
design of cameras for specific robotics tasks.

通过将无导数和基于梯度的优化器结合，利用计算机图形学技术和物理摄像头特性，我们提出了一种新颖的端到端优化方法来为特定机器人任务设计摄像头，通过软件原型设计摄像头、模拟操作环境和任务以及优化摄像头设计，从而提高机器人在其应用中的性能。