Gaze object prediction is a newly proposed task that aims to discover the
objects being stared at by humans. It is of great application significance but
still lacks a unified solution framework. An intuitive solution is to
incorporate an object detection branch into an existing gaze prediction method.
However, previous gaze prediction methods usually use two different networks to
extract features from scene image and head image, which would lead to heavy
network architecture and prevent each branch from joint optimization. In this
paper, we build a novel framework named GaTector to tackle the gaze object
prediction problem in a unified way. Particularly, a specific-general-specific
(SGS) feature extractor is firstly proposed to utilize a shared backbone to
extract general features for both scene and head images. To better consider the
specificity of inputs and tasks, SGS introduces two input-specific blocks
before the shared backbone and three task-specific blocks after the shared
backbone. Specifically, a novel Defocus layer is designed to generate
object-specific features for the object detection task without losing
information or requiring extra computations. Moreover, the energy aggregation
loss is introduced to guide the gaze heatmap to concentrate on the stared box.
In the end, we propose a novel wUoC metric that can reveal the difference
between boxes even when they share no overlapping area. Extensive experiments
on the GOO dataset verify the superiority of our method in all three tracks,
i.e. object detection, gaze estimation, and gaze object prediction.

本研究提出了一种名为 GaTector 的新型框架，它使用了一个特定 - 通用 - 特定的特征提取器，并引入了能量聚合损失和 wUoC 度量标准，以在统一的框架下解决凝视目标预测问题。实验证明，该方法在目标检测、凝视估计和凝视对象预测三个任务中都具有优异的性能。