Progress in human behavior modeling involves understanding both implicit,
early-stage perceptual behavior such as human attention and explicit,
later-stage behavior such as subjective ratings/preferences. Yet, most prior
research has focused on modeling implicit and explicit human behavior in
isolation. Can we build a unified model of human attention and preference
behavior that reliably works across diverse types of visual content? Such a
model would enable predicting subjective feedback such as overall satisfaction
or aesthetic quality ratings, along with the underlying human attention or
interaction heatmaps and viewing order, enabling designers and content-creation
models to optimize their creation for human-centric improvements. In this
paper, we propose UniAR -- a unified model that predicts both implicit and
explicit human behavior across different types of visual content. UniAR
leverages a multimodal transformer, featuring distinct prediction heads for
each facet, and predicts attention heatmap, scanpath or viewing order, and
subjective rating/preference. We train UniAR on diverse public datasets
spanning natural images, web pages and graphic designs, and achieve leading
performance on multiple benchmarks across different image domains and various
behavior modeling tasks. Potential applications include providing instant
feedback on the effectiveness of UIs/digital designs/images, and serving as a
reward model to further optimize design/image creation.

人类行为建模方面取得的进展涉及对隐含的、早期的感知行为（如注意力）以及明确的、后期的行为（如主观评分 / 喜好）的理解。然而，大部分先前的研究都集中在隔离地建模隐含和明确的人类行为上。我们能否建立一个统一的人类注意力和偏好行为模型，可在各种类型的视觉内容中可靠地工作？这样的模型将能够预测主观反馈，如整体满意度或审美质量评级，以及潜在的人类注意力或互动热图和浏览顺序，从而使设计师和内容创作模型能够优化其创作以实现以人为中心的改进。在本文中，我们提出了 UniAR - 一个统一的模型，可以在不同类型的视觉内容上预测隐含和明确的人类行为。UniAR 利用了一种多模态变换器，具有每个方面的不同预测头，并预测注意力热图、扫描路径或浏览顺序，以及主观评级 / 喜好。我们在涵盖自然图像、网页和图形设计的多样公共数据集上训练 UniAR，并在不同图像领域和各种行为建模任务上取得了领先的性能。潜在应用包括即时提供对界面 / UI 设计 / 图像有效性的反馈，并作为奖励模型进一步优化设计 / 图像创作。