We propose an end-to-end approach for gaze target detection: predicting a
head-target connection between individuals and the target image regions they
are looking at. Most of the existing methods use independent components such as
off-the-shelf head detectors or have problems in establishing associations
between heads and gaze targets. In contrast, we investigate an end-to-end
multi-person Gaze target detection framework with Heads and Targets Association
(GazeHTA), which predicts multiple head-target instances based solely on input
scene image. GazeHTA addresses challenges in gaze target detection by (1)
leveraging a pre-trained diffusion model to extract scene features for rich
semantic understanding, (2) re-injecting a head feature to enhance the head
priors for improved head understanding, and (3) learning a connection map as
the explicit visual associations between heads and gaze targets. Our extensive
experimental results demonstrate that GazeHTA outperforms state-of-the-art gaze
target detection methods and two adapted diffusion-based baselines on two
standard datasets.

我们提出了一种端到端的方法来检测凝视目标：预测个体和他们正在注视的目标图像区域之间的头目标连接。我们的实验结果表明，GazeHTA 在两个标准数据集上优于最先进的凝视目标检测方法和两个改进的基于扩散的基准模型。

GazeHTA: 基于头部 - 目标关联的端到端凝视目标检测

GazeHTA: End-to-end Gaze Target Detection with Head-Target Association

Gaze target detection aims to predict the image location where the person is
looking and the probability that a gaze is out of the scene. Several works have
tackled this task by regressing a gaze heatmap centered on the gaze location,
however, they overlooked decoding the relationship between the people and the
gazed objects. This paper proposes a Transformer-based architecture that
automatically detects objects (including heads) in the scene to build
associations between every head and the gazed-head/object, resulting in a
comprehensive, explainable gaze analysis composed of: gaze target area, gaze
pixel point, the class and the image location of the gazed-object. Upon
evaluation of the in-the-wild benchmarks, our method achieves state-of-the-art
results on all metrics (up to 2.91% gain in AUC, 50% reduction in gaze
distance, and 9% gain in out-of-frame average precision) for gaze target
detection and 11-13% improvement in average precision for the classification
and the localization of the gazed-objects. The code of the proposed method is
available this https URL

该研究使用基于 Transformer 的架构自动检测图像中的对象，并建立对象与注视的关联，从而实现全面的、可解释的注视分析，包括注视目标区域、注视像素点、被注视对象的类别和图像位置。该方法在各项指标上均取得了最新的成果，对于注视目标检测提高了 AUC 的达到 2.91%、注视距离减少了 50%、注视对象分类和定位平均精度提高了 11-13%，且代码可在链接中获得。

目标感知的凝视目标检测

Object-aware Gaze Target Detection

This paper addresses the gaze target detection problem in single images
captured from the third-person perspective. We present a multimodal deep
architecture to infer where a person in a scene is looking. This spatial model
is trained on the head images of the person-of- interest, scene and depth maps
representing rich context information. Our model, unlike several prior art, do
not require supervision of the gaze angles, do not rely on head orientation
information and/or location of the eyes of person-of-interest. Extensive
experiments demonstrate the stronger performance of our method on multiple
benchmark datasets. We also investigated several variations of our method by
altering joint-learning of multimodal data. Some variations outperform a few
prior art as well. First time in this paper, we inspect domain adaption for
gaze target detection, and we empower our multimodal network to effectively
handle the domain gap across datasets. The code of the proposed method is
available at
this https URL.

本研究讨论了单幅图像的凝视目标检测问题，提出了使用多模态深度架构来推断场景中人物的注视位置的方法，并考虑了不同数据集之间的领域适应。