Progress in human behavior modeling involves understanding both implicit,
early-stage perceptual behavior such as human attention and explicit,
later-stage behavior such as subjective ratings/preferences. Yet, most prior
research has focused on modeling implicit and explicit human behavior in
isolation. Can we build a unified model of human attention and preference
behavior that reliably works across diverse types of visual content? Such a
model would enable predicting subjective feedback such as overall satisfaction
or aesthetic quality ratings, along with the underlying human attention or
interaction heatmaps and viewing order, enabling designers and content-creation
models to optimize their creation for human-centric improvements. In this
paper, we propose UniAR -- a unified model that predicts both implicit and
explicit human behavior across different types of visual content. UniAR
leverages a multimodal transformer, featuring distinct prediction heads for
each facet, and predicts attention heatmap, scanpath or viewing order, and
subjective rating/preference. We train UniAR on diverse public datasets
spanning natural images, web pages and graphic designs, and achieve leading
performance on multiple benchmarks across different image domains and various
behavior modeling tasks. Potential applications include providing instant
feedback on the effectiveness of UIs/digital designs/images, and serving as a
reward model to further optimize design/image creation.

人类行为建模方面取得的进展涉及对隐含的、早期的感知行为（如注意力）以及明确的、后期的行为（如主观评分 / 喜好）的理解。然而，大部分先前的研究都集中在隔离地建模隐含和明确的人类行为上。我们能否建立一个统一的人类注意力和偏好行为模型，可在各种类型的视觉内容中可靠地工作？这样的模型将能够预测主观反馈，如整体满意度或审美质量评级，以及潜在的人类注意力或互动热图和浏览顺序，从而使设计师和内容创作模型能够优化其创作以实现以人为中心的改进。在本文中，我们提出了 UniAR - 一个统一的模型，可以在不同类型的视觉内容上预测隐含和明确的人类行为。UniAR 利用了一种多模态变换器，具有每个方面的不同预测头，并预测注意力热图、扫描路径或浏览顺序，以及主观评级 / 喜好。我们在涵盖自然图像、网页和图形设计的多样公共数据集上训练 UniAR，并在不同图像领域和各种行为建模任务上取得了领先的性能。潜在应用包括即时提供对界面 / UI 设计 / 图像有效性的反馈，并作为奖励模型进一步优化设计 / 图像创作。

UniAR：统一人类视觉内容中的注意力与反应预测

UniAR: Unifying Human Attention and Response Prediction on Visual  Content

Mobile health applications, including those that track activities such as
exercise, sleep, and diet, are becoming widely used. Accurately predicting
human actions is essential for targeted recommendations that could improve our
health and for personalization of these applications. However, making such
predictions is extremely difficult due to the complexities of human behavior,
which consists of a large number of potential actions that vary over time,
depend on each other, and are periodic. Previous work has not jointly modeled
these dynamics and has largely focused on item consumption patterns instead of
broader types of behaviors such as eating, commuting or exercising. In this
work, we develop a novel statistical model for Time-varying, Interdependent,
and Periodic Action Sequences. Our approach is based on personalized,
multivariate temporal point processes that model time-varying action
propensities through a mixture of Gaussian intensities. Our model captures
short-term and long-term periodic interdependencies between actions through
Hawkes process-based self-excitations. We evaluate our approach on two activity
logging datasets comprising 12 million actions taken by 20 thousand users over
17 months. We demonstrate that our approach allows us to make successful
predictions of future user actions and their timing. Specifically, our model
improves predictions of actions, and their timing, over existing methods across
multiple datasets by up to 156%, and up to 37%, respectively. Performance
improvements are particularly large for relatively rare and periodic actions
such as walking and biking, improving over baselines by up to 256%. This
demonstrates that explicit modeling of dependencies and periodicities in
real-world behavior enables successful predictions of future actions, with
implications for modeling human behavior, app personalization, and targeting of
health interventions.

本文开发了一种基于个性化、多变量时间点过程的新型统计模型，用于对移动健康应用中的人类行为建模，以得出未来用户行为及其时机的准确预测。