Robot Imitation Learning (IL) is a widely used method for training robots to
perform manipulation tasks that involve mimicking human demonstrations to
acquire skills. However, its practicality has been limited due to its
requirement that users be trained in operating real robot arms to provide
demonstrations. This paper presents an innovative solution: an Augmented
Reality (AR)-assisted framework for demonstration collection, empowering
non-roboticist users to produce demonstrations for robot IL using devices like
the HoloLens 2. Our framework facilitates scalable and diverse demonstration
collection for real-world tasks. We validate our approach with experiments on
three classical robotics tasks: reach, push, and pick-and-place. The real robot
performs each task successfully while replaying demonstrations collected via
AR.

通过增强现实技术辅助的演示获取框架，该论文提出了一种创新的解决方案，使非机器人专业人员能够使用 HoloLens 2 等设备为机器人模仿学习提供演示，从而实现了可扩展和多样化的真实世界任务演示收集。通过在三个经典机器人任务（到达、推动和拾取放置）上进行实验证明了该方法的有效性，实际机器人能够成功执行每个任务并回放通过增强现实收集的演示。

可扩展机器人模仿学习的增强现实演示

Augmented Reality Demonstrations for Scalable Robot Imitation Learning

Training helps to maintain and improve sufficient muscle function, body
control, and body coordination. These are important to reduce the risk of
fracture incidents caused by falls, especially for the elderly or people
recovering from injury. Virtual reality training can offer a cost-effective and
individualized training experience. We present an application for the HoloLens
2 to enable musculoskeletal training for elderly and impaired persons to allow
for autonomous training and automatic progress evaluation. We designed a
virtual downhill skiing scenario that is controlled by body movement to
stimulate balance and body control. By adapting the parameters of the ski
slope, we can tailor the intensity of the training to individual users. In this
work, we evaluate whether the movement data of the HoloLens 2 alone is
sufficient to control and predict body movement and joint angles during
musculoskeletal training. We record the movements of 10 healthy volunteers with
external tracking cameras and track a set of body and joint angles of the
participant during training. We estimate correlation coefficients and
systematically analyze whether whole body movement can be derived from the
movement data of the HoloLens 2. No participant reports movement sickness
effects and all were able to quickly interact and control their movement during
skiing. Our results show a high correlation between HoloLens 2 movement data
and the external tracking of the upper body movement and joint angles of the
lower limbs.

我们设计了一个基于 HoloLens 2 的应用程序，通过自主训练和自动评估来实现老年人和受伤人员的肌肉骨骼训练，我们评估了 HoloLens 2 的运动数据是否足以控制和预测肌肉骨骼训练中的身体动作和关节角度，结果显示 HoloLens 2 的运动数据与上半身运动和下肢关节角度的外部跟踪存在高度相关性。

基于虚拟现实的身体追踪促进肌肉骨骼训练

VR-based body tracking to stimulate musculoskeletal training

A novel method to control the Spot robot of Boston Dynamics by Hololens 2 is
proposed. This method is mainly designed for people with physical disabilities,
users can control the robot's movement and robot arm without using their hands.
The eye gaze tracking and head motion tracking technologies of Hololens 2 are
utilized for sending control commands. The movement of the robot would follow
the eye gaze and the robot arm would mimic the pose of the user's head. Through
our experiment, our method is comparable with the traditional control method by
joystick in both time efficiency and user experience. Demo can be found on our
project webpage: this https URL

本研究提出一种通过 Hololens 2 控制 Boston Dynamics 的 Spot 机器人的新方法，该方法主要面向残障人士，用户可以通过眼神追踪和头部动作追踪等技术来控制机器人的运动和机械臂的姿势，与传统的操纵杆控制方法相比具有更好的用户体验和时间效率。