Adversarial imitation learning has become a widely used imitation learning
framework. The discriminator is often trained by taking expert demonstrations
and policy trajectories as examples respectively from two categories (positive
vs. negative) and the policy is then expected to produce trajectories that are
indistinguishable from the expert demonstrations. But in the real world, the
collected expert demonstrations are more likely to be imperfect, where only an
unknown fraction of the demonstrations are optimal. Instead of treating
imperfect expert demonstrations as absolutely positive or negative, we
investigate unlabeled imperfect expert demonstrations as they are. A
positive-unlabeled adversarial imitation learning algorithm is developed to
dynamically sample expert demonstrations that can well match the trajectories
from the constantly optimized agent policy. The trajectories of an initial
agent policy could be closer to those non-optimal expert demonstrations, but
within the framework of adversarial imitation learning, agent policy will be
optimized to cheat the discriminator and produce trajectories that are similar
to those optimal expert demonstrations. Theoretical analysis shows that our
method learns from the imperfect demonstrations via a self-paced way.
Experimental results on MuJoCo and RoboSuite platforms demonstrate the
effectiveness of our method from different aspects.

本文提出了一种基于自对抗的无标签专家演示的算法，利用全面学习方法，动态地抽样专家演示，并与不断优化的策略进行比对，可以优化生成的轨迹，实现了动态的自我适应学习。

对抗性模仿学习中的无标签不完美演示

Unlabeled Imperfect Demonstrations in Adversarial Imitation Learning

Over the past two decades, numerous studies have demonstrated how less
predictable (i.e. higher surprisal) words take more time to read. In general,
these previous studies implicitly assumed the reading process to be purely
responsive: readers observe a new word and allocate time to read it as
required. These results, however, are also compatible with a reading time that
is anticipatory: readers could, e.g., allocate time to a future word based on
their expectation about it. In this work, we examine the anticipatory nature of
reading by looking at how people's predictions about upcoming material
influence reading times. Specifically, we test anticipation by looking at the
effects of surprisal and contextual entropy on four reading-time datasets: two
self-paced and two eye-tracking. In three of four datasets tested, we find that
the entropy predicts reading times as well as (or better than) the surprisal.
We then hypothesise four cognitive mechanisms through which the contextual
entropy could impact RTs -- three of which we design experiments to analyse.
Overall, our results support a view of reading that is both anticipatory and
responsive.

本篇研究检验了阅读的预测性质，发现上下文熵对阅读时间的预测能力高于或等于惊奇程度，在四个阅读数据中三个能够预测阅读时间。此外，作者提出了四个认知机制来支持上下文熵对阅读时间的影响，从而支持阅读具有先见性和响应性的观点。