In this work, we introduce FaceXformer, an end-to-end unified transformer
model for a comprehensive range of facial analysis tasks such as face parsing,
landmark detection, head pose estimation, attributes recognition, and
estimation of age, gender, race, and landmarks visibility. Conventional methods
in face analysis have often relied on task-specific designs and preprocessing
techniques, which limit their approach to a unified architecture. Unlike these
conventional methods, our FaceXformer leverages a transformer-based
encoder-decoder architecture where each task is treated as a learnable token,
enabling the integration of multiple tasks within a single framework. Moreover,
we propose a parameter-efficient decoder, FaceX, which jointly processes face
and task tokens, thereby learning generalized and robust face representations
across different tasks. To the best of our knowledge, this is the first work to
propose a single model capable of handling all these facial analysis tasks
using transformers. We conducted a comprehensive analysis of effective
backbones for unified face task processing and evaluated different task queries
and the synergy between them. We conduct experiments against state-of-the-art
specialized models and previous multi-task models in both intra-dataset and
cross-dataset evaluations across multiple benchmarks. Additionally, our model
effectively handles images "in-the-wild," demonstrating its robustness and
generalizability across eight different tasks, all while maintaining the
real-time performance of 37 FPS.

我们介绍了 FaceXformer，这是一个端到端的统一变换器模型，用于综合范围的面部分析任务，如面部解析、标志检测、头部姿势估计、属性识别以及年龄、性别、种族和标志点可见性的估计。我们的 FaceXformer 利用基于变换器的编码器 - 解码器架构，将每个任务视为可学习的标记，从而在单一框架内集成多个任务。此外，我们提出了一个参数高效的解码器 FaceX，它共同处理面部和任务标记，从而在不同任务之间学习通用和鲁棒的面部表示。根据我们的了解，这是首次使用变换器提出了一个能够处理所有这些面部分析任务的单一模型。我们对统一面部任务处理的有效骨干进行了全面分析，并评估了不同任务查询和它们之间的协同作用。我们在多个基准测试中进行了对比实验证实，与最先进的专用模型和之前的多任务模型相比，在数据集内和跨数据集评估中，我们的模型均表现出色。此外，我们的模型有效处理来自 “野外” 的图像，展示了它在八个不同任务上的鲁棒性和普适性，同时保持了每秒 37 帧的实时性能。