Recently, neural control policies have outperformed existing model-based
planning-and-control methods for autonomously navigating quadrotors through
cluttered environments in minimum time. However, they are not perception aware,
a crucial requirement in vision-based navigation due to the camera's limited
field of view and the underactuated nature of a quadrotor. We propose a
learning-based system that achieves perception-aware, agile flight in cluttered
environments. Our method combines imitation learning with reinforcement
learning (RL) by leveraging a privileged learning-by-cheating framework. Using
RL, we first train a perception-aware teacher policy with full-state
information to fly in minimum time through cluttered environments. Then, we use
imitation learning to distill its knowledge into a vision-based student policy
that only perceives the environment via a camera. Our approach tightly couples
perception and control, showing a significant advantage in computation speed
(10 times faster) and success rate. We demonstrate the closed-loop control
performance using hardware-in-the-loop simulation.

提出了一种基于学习的系统，通过模仿学习和强化学习的结合，实现了感知感知，在杂乱环境中实现了敏捷的飞行，并通过闭环控制演示了其性能。

在杂乱环境中学习感知敏捷飞行

Learning Perception-Aware Agile Flight in Cluttered Environments

We describe a learning-based system that estimates the camera position and
orientation from a single input image relative to a known environment. The
system is flexible w.r.t. the amount of information available at test and at
training time, catering to different applications. Input images can be RGB-D or
RGB, and a 3D model of the environment can be utilized for training but is not
necessary. In the minimal case, our system requires only RGB images and ground
truth poses at training time, and it requires only a single RGB image at test
time. The framework consists of a deep neural network and fully differentiable
pose optimization. The neural network predicts so called scene coordinates,
i.e. dense correspondences between the input image and 3D scene space of the
environment. The pose optimization implements robust fitting of pose parameters
using differentiable RANSAC (DSAC) to facilitate end-to-end training. The
system, an extension of DSAC++ and referred to as DSAC*, achieves
state-of-the-art accuracy an various public datasets for RGB-based
re-localization, and competitive accuracy for RGB-D-based re-localization.

这篇论文介绍了一种基于学习的系统，可以通过单个输入图像相对于已知环境估计相机的位置和方向，该系统采用了深度神经网络和可完全微分的姿态优化，可以适应不同应用程序，包括使用 RGB-D 或 RGB 图像进行训练，并可以使用环境的三维模型进行训练，但不是必需品。

使用 DSAC 从 RGB 和 RGB-D 图像中进行视觉相机重定位

Visual Camera Re-Localization from RGB and RGB-D Images Using DSAC

Manipulation of deformable objects, such as ropes and cloth, is an important
but challenging problem in robotics. We present a learning-based system where a
robot takes as input a sequence of images of a human manipulating a rope from
an initial to goal configuration, and outputs a sequence of actions that can
reproduce the human demonstration, using only monocular images as input. To
perform this task, the robot learns a pixel-level inverse dynamics model of
rope manipulation directly from images in a self-supervised manner, using about
60K interactions with the rope collected autonomously by the robot. The human
demonstration provides a high-level plan of what to do and the low-level
inverse model is used to execute the plan. We show that by combining the high
and low-level plans, the robot can successfully manipulate a rope into a
variety of target shapes using only a sequence of human-provided images for
direction.

这篇论文提出了一种基于学习的系统，通过自监督学习从图像中学习绳索操纵的像素级逆动力学模型，并使用高低级计划成功地模仿人类示范，将可变形物体如绳索和布料进行操纵。