Most existing policy learning solutions require the learning agents to
receive high-quality supervision signals such as well-designed rewards in
reinforcement learning (RL) or high-quality expert demonstrations in behavioral
cloning (BC). These quality supervisions are usually infeasible or
prohibitively expensive to obtain in practice. We aim for a unified framework
that leverages the available cheap weak supervisions to perform policy learning
efficiently. To handle this problem, we treat the "weak supervision" as
imperfect information coming from a peer agent, and evaluate the learning
agent's policy based on a "correlated agreement" with the peer agent's policy
(instead of simple agreements). Our approach explicitly punishes a policy for
overfitting to the weak supervision. In addition to theoretical guarantees,
extensive evaluations on tasks including RL with noisy rewards, BC with weak
demonstrations, and standard policy co-training show that our method leads to
substantial performance improvements, especially when the complexity or the
noise of the learning environments is high.

本研究提出了一种基于弱监督学习的框架，通过与所有前置学习 agent 的策略建立相关协议，惩罚过拟合，并在弱监督下对 RL、BC 及标准政策联合训练等任务中得到了显著的性能提升。