We tackle the problem of learning complex, general behaviors directly in the
real world. We propose an approach for robots to efficiently learn manipulation
skills using only a handful of real-world interaction trajectories from many
different settings. Inspired by the success of learning from large-scale
datasets in the fields of computer vision and natural language, our belief is
that in order to efficiently learn, a robot must be able to leverage
internet-scale, human video data. Humans interact with the world in many
interesting ways, which can allow a robot to not only build an understanding of
useful actions and affordances but also how these actions affect the world for
manipulation. Our approach builds a structured, human-centric action space
grounded in visual affordances learned from human videos. Further, we train a
world model on human videos and fine-tune on a small amount of robot
interaction data without any task supervision. We show that this approach of
affordance-space world models enables different robots to learn various
manipulation skills in complex settings, in under 30 minutes of interaction.
Videos can be found at this https URL

我们提出了一种方法，通过仅利用许多不同设置的少量实际交互轨迹，使机器人能够有效地学习操纵技能，并由此建立了一个基于人的视频的结构化行为空间，从而使不同的机器人能够在复杂环境下学习各种操纵技能。

从人类视频中构建结构化世界模型

Structured World Models from Human Videos

Understanding human affective behaviour, especially in the dynamics of
real-world settings, requires Facial Expression Recognition (FER) models to
continuously adapt to individual differences in user expression, contextual
attributions, and the environment. Current (deep) Machine Learning (ML)-based
FER approaches pre-trained in isolation on benchmark datasets fail to capture
the nuances of real-world interactions where data is available only
incrementally, acquired by the agent or robot during interactions. New learning
comes at the cost of previous knowledge, resulting in catastrophic forgetting.
Lifelong or Continual Learning (CL), on the other hand, enables adaptability in
agents by being sensitive to changing data distributions, integrating new
information without interfering with previously learnt knowledge. Positing CL
as an effective learning paradigm for FER, this work presents the Continual
Facial Expression Recognition (ConFER) benchmark that evaluates popular CL
techniques on FER tasks. It presents a comparative analysis of several CL-based
approaches on popular FER datasets such as CK+, RAF-DB, and AffectNet and
present strategies for a successful implementation of ConFER for Affective
Computing (AC) research. CL techniques, under different learning settings, are
shown to achieve state-of-the-art (SOTA) performance across several datasets,
thus motivating a discussion on the benefits of applying CL principles towards
human behaviour understanding, particularly from facial expressions, as well
the challenges entailed.

本文提出了 Continual Facial Expression Recognition（ConFER）基准测试，评估了不同的 Continual Learning（CL）方法在面部表情识别任务中的性能，表明 CL 技术在不同的学习设置下能够实现多个数据集上的最先进的性能，因此促进了关于将 CL 原则应用于人类面部表情的行为理解中的益处和挑战的讨论。

持续面部表情识别：基准测试

Continual Facial Expression Recognition: A Benchmark

Tactile predictive models can be useful across several robotic manipulation
tasks, e.g. robotic pushing, robotic grasping, slip avoidance, and in-hand
manipulation. However, available tactile prediction models are mostly studied
for image-based tactile sensors and there is no comparison study indicating the
best performing models. In this paper, we presented two novel data-driven
action-conditioned models for predicting tactile signals during real-world
physical robot interaction tasks (1) action condition tactile prediction and
(2) action conditioned tactile-video prediction models. We use a magnetic-based
tactile sensor that is challenging to analyse and test state-of-the-art
predictive models and the only existing bespoke tactile prediction model. We
compare the performance of these models with those of our proposed models. We
perform the comparison study using our novel tactile enabled dataset containing
51,000 tactile frames of a real-world robotic manipulation task with 11
flat-surfaced household objects. Our experimental results demonstrate the
superiority of our proposed tactile prediction models in terms of qualitative,
quantitative and slip prediction scores.

本研究提出了两种新型的数据驱动的行动条件触觉预测模型，用于预测真实世界中机器人交互任务期间的触觉信号，并使用我们的新颖触觉数据集对这些模型进行了比较，结果显示我们的模型在定性、定量和滑动预测得分方面都具有优越性。