We propose a novel system for action sequence planning based on a combination
of affordance recognition and a neural forward model predicting the effects of
affordance execution. By performing affordance recognition on predicted
futures, we avoid reliance on explicit affordance effect definitions for
multi-step planning. Because the system learns affordance effects from
experience data, the system can foresee not just the canonical effects of an
affordance, but also situation-specific side-effects. This allows the system to
avoid planning failures due to such non-canonical effects, and makes it
possible to exploit non-canonical effects for realising a given goal. We
evaluate the system in simulation, on a set of test tasks that require
consideration of canonical and non-canonical affordance effects.

提出了一种基于 affordance 识别和神经前向模型预测 affordance 执行效果的行动序列规划系统，可以避免对多步骤计划的明确 affordance 效果定义的依赖。通过对预测未来的 affordance 识别，系统可以从经验数据中学习 affordance 效果，并且可以预见特定情况的副作用，避免了由于非规范效果引起的计划失败。并且可以利用非规范效果实现特定目标。在仿真系统上测试了该系统，在需要考虑规范和非规范 affordance 效果的一组测试任务上进行评估。

识别预测未来中的能 affordance 并考虑非规范 affordance 影响进行规划

Recognising Affordances in Predicted Futures to Plan with Consideration of Non-canonical Affordance Effects

Object affordance is an important concept in human-object interaction,
providing information on action possibilities based on human motor capacity and
objects' physical property thus benefiting tasks such as action anticipation
and robot imitation learning. However, existing datasets often: 1) mix up
affordance with object functionality; 2) confuse affordance with goal-related
action; and 3) ignore human motor capacity. This paper proposes an efficient
annotation scheme to address these issues by combining goal-irrelevant motor
actions and grasp types as affordance labels and introducing the concept of
mechanical action to represent the action possibilities between two objects. We
provide new annotations by applying this scheme to the EPIC-KITCHENS dataset
and test our annotation with tasks such as affordance recognition. We
qualitatively verify that models trained with our annotation can distinguish
affordance and mechanical actions.

本文提出了一种有效的注释方案，通过将目标无关的运动动作和抓握类型作为能力标签，并引入机械动作概念来描述两个物体之间的动作可能性，解决了现有数据集中的问题，并通过将此方案应用于 EPIC-KITCHENS 数据集来验证了模型的有效性。

对自我中心动作视频数据集的精准可负担注释

Precise Affordance Annotation for Egocentric Action Video Datasets

The physical and textural attributes of objects have been widely studied for
recognition, detection and segmentation tasks in computer vision. A number of
datasets, such as large scale ImageNet, have been proposed for feature learning
using data hungry deep neural networks and for hand-crafted feature extraction.
To intelligently interact with objects, robots and intelligent machines need
the ability to infer beyond the traditional physical/textural attributes, and
understand/learn visual cues, called visual affordances, for affordance
recognition, detection and segmentation. To date there is no publicly available
large dataset for visual affordance understanding and learning. In this paper,
we introduce a large scale multi-view RGBD visual affordance learning dataset,
a benchmark of 47210 RGBD images from 37 object categories, annotated with 15
visual affordance categories and 35 cluttered/complex scenes with different
objects and multiple affordances. To the best of our knowledge, this is the
first ever and the largest multi-view RGBD visual affordance learning dataset.
We benchmark the proposed dataset for affordance recognition and segmentation.
To achieve this we propose an Affordance Recognition Network a.k.a ARNet. In
addition, four state-of-the-art deep learning networks are evaluated for
affordance segmentation task. Our experimental results showcase the challenging
nature of the dataset and present definite prospects for new and robust
affordance learning algorithms. The dataset is available at:
this https URL.

本文介绍了一个大规模多视角 RGBD 视觉可供性学习数据集及其基准测试方法，它包含了来自 37 种对象类别、配有 15 种视觉可供性类别和 35 种杂乱 / 复杂的场景的 47210 个 RGBD 图像。我们提出了一个名为 ARNet 的【可供性识别网络】，证明了该数据集的挑战性以及提出的新的强大的可供性学习算法的明确前景。