Simulation-to-reality (sim-to-real) transfer is a fundamental problem for
robot learning. Domain Randomization, which adds randomization during training,
is a powerful technique that effectively addresses the sim-to-real gap.
However, the noise in observations makes learning significantly harder.
Recently, studies have shown that employing a teacher-student learning paradigm
can accelerate training in randomized environments. Learned with privileged
information, a teacher agent can instruct the student agent to operate in noisy
environments. However, this approach is often not sample efficient as the
experience collected by the teacher is discarded completely when training the
student, wasting information revealed by the environment. In this work, we
extend the teacher-student learning paradigm by proposing a sample efficient
learning framework termed Learn to Teach (L2T) that recycles experience
collected by the teacher agent. We observe that the dynamics of the
environments for both agents remain unchanged, and the state space of the
teacher is coupled with the observation space of the student. We show that a
single-loop algorithm can train both the teacher and student agents under both
Reinforcement Learning and Inverse Reinforcement Learning contexts. We
implement variants of our methods, conduct experiments on the MuJoCo benchmark,
and apply our methods to the Cassie robot locomotion problem. Extensive
experiments show that our method achieves competitive performance while only
requiring environmental interaction with the teacher.

模拟到现实的迁移是机器人学习的一个基本问题，本文提出了一种学习框架，通过引入师生学习范式和样本高效性，解决了培训中噪声观测造成的困难，实现了模拟环境下机器人的高性能表现。

学习教学：提高师生学习中的样本效率，用于模拟到真实场景的迁移

Learn to Teach: Improve Sample Efficiency in Teacher-student Learning  for Sim-to-Real Transfer

We study the problem of robotic stacking with objects of complex geometry. We
propose a challenging and diverse set of such objects that was carefully
designed to require strategies beyond a simple "pick-and-place" solution. Our
method is a reinforcement learning (RL) approach combined with vision-based
interactive policy distillation and simulation-to-reality transfer. Our learned
policies can efficiently handle multiple object combinations in the real world
and exhibit a large variety of stacking skills. In a large experimental study,
we investigate what choices matter for learning such general vision-based
agents in simulation, and what affects optimal transfer to the real robot. We
then leverage data collected by such policies and improve upon them with
offline RL. A video and a blog post of our work are provided as supplementary
material.

本文使用增强学习方法和视觉技术在真实环境中完成对复杂物体进行叠放的任务，重点研究了这种方法的优越性和训练问题。