Imitation Learning is a promising paradigm for learning complex robot
manipulation skills by reproducing behavior from human demonstrations. However,
manipulation tasks often contain bottleneck regions that require a sequence of
precise actions to make meaningful progress, such as a robot inserting a pod
into a coffee machine to make coffee. Trained policies can fail in these
regions because small deviations in actions can lead the policy into states not
covered by the demonstrations. Intervention-based policy learning is an
alternative that can address this issue -- it allows human operators to monitor
trained policies and take over control when they encounter failures. In this
paper, we build a data collection system tailored to 6-DoF manipulation
settings, that enables remote human operators to monitor and intervene on
trained policies. We develop a simple and effective algorithm to train the
policy iteratively on new data collected by the system that encourages the
policy to learn how to traverse bottlenecks through the interventions. We
demonstrate that agents trained on data collected by our intervention-based
system and algorithm outperform agents trained on an equivalent number of
samples collected by non-interventional demonstrators, and further show that
our method outperforms multiple state-of-the-art baselines for learning from
the human interventions on a challenging robot threading task and a coffee
making task. Additional results and videos at
this https URL .

本文介绍使用干预性策略学习的方法来解决机器人操作任务中必须经过精确定序的地方的问题，提出一种 6 自由度机器人操作任务的数据采集系统，并开发了一个简单而有效的算法来收集新数据以遍历通过这些难点，使用干预策略学习的代理在机器人的线路穿线任务和制造咖啡任务中的表现优于其他多种基线算法。

使用远程遥操作的人在循环中模仿学习

Human-in-the-Loop Imitation Learning using Remote Teleoperation

One major problem with cyberbullying research is the lack of data, since
researchers are traditionally forced to rely on survey data where victims and
perpetrators self-report their impressions. In this paper, an automatic data
collection system is presented that continuously collects in-game chat data
from one of the most popular online multi-player games: World of Tanks. The
data was collected and combined with other information about the players from
available online data services. It presents a scoring scheme to enable
identification of cyberbullying based on current research. Classification of
the collected data was carried out using simple feature detection with SQL
database queries and compared to classification from AI-based sentiment text
analysis services that have recently become available and further against
manually classified data using a custom-built classification client built for
this paper. The simple SQL classification proved to be quite useful at
identifying some features of toxic chat such as the use of bad language or
racist sentiments, however the classification by the more sophisticated online
sentiment analysis services proved to be disappointing. The results were then
examined for insights into cyberbullying within this game and it was shown that
it should be possible to reduce cyberbullying within the World of Tanks game by
a significant factor by simply freezing the player's ability to communicate
through the in-game chat function for a short period after the player is killed
within a match. It was also shown that very new players are much less likely to
engage in cyberbullying, suggesting that it may be a learned behaviour from
other players.

本文介绍了一种自动数据收集系统，用于从在线多人游戏《坦克世界》中连续收集游戏聊天数据，并结合在线数据服务的其他信息。并提出了一种评分方案，用于基于当前研究的网络欺凌识别。该研究使用简单的特征检测，通过 SQL 数据库查询对收集的数据进行了分类，结果显示：虽然 SQL 分类很有用，但基于更复杂的在线情感分析服务的分类则不尽人意。最后，研究结果揭示出在游戏中，可以通过冻结玩家在游戏聊天功能中的通信能力，显著减少网络欺凌。同时，新手玩家很少进行网络欺凌，这意味着这些行为可能来自其他玩家的学习。