Affordance grounding, a task to ground (i.e., localize) action possibility
region in objects, which faces the challenge of establishing an explicit link
with object parts due to the diversity of interactive affordance. Human has the
ability that transform the various exocentric interactions to invariant
egocentric affordance so as to counter the impact of interactive diversity. To
empower an agent with such ability, this paper proposes a task of affordance
grounding from exocentric view, i.e., given exocentric human-object interaction
and egocentric object images, learning the affordance knowledge of the object
and transferring it to the egocentric image using only the affordance label as
supervision. To this end, we devise a cross-view knowledge transfer framework
that extracts affordance-specific features from exocentric interactions and
enhances the perception of affordance regions by preserving affordance
correlation. Specifically, an Affordance Invariance Mining module is devised to
extract specific clues by minimizing the intra-class differences originated
from interaction habits in exocentric images. Besides, an Affordance
Co-relation Preserving strategy is presented to perceive and localize
affordance by aligning the co-relation matrix of predicted results between the
two views. Particularly, an affordance grounding dataset named AGD20K is
constructed by collecting and labeling over 20K images from 36 affordance
categories. Experimental results demonstrate that our method outperforms the
representative models in terms of objective metrics and visual quality. Code:
github.com/lhc1224/Cross-View-AG.

提出了一种从超现实视角进行可支配特定元素分割的任务，旨在通过挖掘超现实交互中的特定线索，从而能够仅利用支配标签来学习对于超现实图像中特定物体可支配性的知识，并将其传递到以自我的物体图像为中心的视角中。

从非视角图像中学习 affordance

Learning Affordance Grounding from Exocentric Images

Cross-view image generation has been recently proposed to generate images of
one view from another dramatically different view. In this paper, we
investigate exocentric (third-person) view to egocentric (first-person) view
image generation. This is a challenging task since egocentric view sometimes is
remarkably different from exocentric view. Thus, transforming the appearances
across the two views is a non-trivial task. To this end, we propose a novel
Parallel Generative Adversarial Network (P-GAN) with a novel cross-cycle loss
to learn the shared information for generating egocentric images from
exocentric view. We also incorporate a novel contextual feature loss in the
learning procedure to capture the contextual information in images. Extensive
experiments on the Exo-Ego datasets show that our model outperforms the
state-of-the-art approaches.

本研究提出了一种基于 Parallel Generative Adversarial Network (P-GAN) 和新的交叉环路损失 Cross-cycle Loss 以及新的上下文特征损失 Contextual Feature Loss 的方法来生成第一人称视角的图像，实验证明该方法在 Exo-Ego 数据集上优于现有的方法。