This paper proposes a novel Unified Feature Optimization (UFO) paradigm for
training and deploying deep models under real-world and large-scale scenarios,
which requires a collection of multiple AI functions. UFO aims to benefit each
single task with a large-scale pretraining on all tasks. Compared with the well
known foundation model, UFO has two different points of emphasis, i.e.,
relatively smaller model size and NO adaptation cost: 1) UFO squeezes a wide
range of tasks into a moderate-sized unified model in a multi-task learning
manner and further trims the model size when transferred to down-stream tasks.
2) UFO does not emphasize transfer to novel tasks. Instead, it aims to make the
trimmed model dedicated for one or more already-seen task. With these two
characteristics, UFO provides great convenience for flexible deployment, while
maintaining the benefits of large-scale pretraining. A key merit of UFO is that
the trimming process not only reduces the model size and inference consumption,
but also even improves the accuracy on certain tasks. Specifically, UFO
considers the multi-task training and brings two-fold impact on the unified
model: some closely related tasks have mutual benefits, while some tasks have
conflicts against each other. UFO manages to reduce the conflicts and to
preserve the mutual benefits through a novel Network Architecture Search (NAS)
method. Experiments on a wide range of deep representation learning tasks
(i.e., face recognition, person re-identification, vehicle re-identification
and product retrieval) show that the model trimmed from UFO achieves higher
accuracy than its single-task-trained counterpart and yet has smaller model
size, validating the concept of UFO. Besides, UFO also supported the release of
17 billion parameters computer vision (CV) foundation model which is the
largest CV model in the industry.

该论文提出了一种新的统一特征优化（UFO）模型，结合多任务学习、网络架构搜索等方法，极大方便模型的灵活部署，在保持大规模预训练精度的基础上，实现了模型体积缩小和预测精度提高。