This paper aims to establish a generic multi-modal foundation model that has
the scalable capability to massive downstream applications in E-commerce.
Recently, large-scale vision-language pretraining approaches have achieved
remarkable advances in the general domain. However, due to the significant
differences between natural and product images, directly applying these
frameworks for modeling image-level representations to E-commerce will be
inevitably sub-optimal. To this end, we propose an instance-centric multi-modal
pretraining paradigm called ECLIP in this work. In detail, we craft a decoder
architecture that introduces a set of learnable instance queries to explicitly
aggregate instance-level semantics. Moreover, to enable the model to focus on
the desired product instance without reliance on expensive manual annotations,
two specially configured pretext tasks are further proposed. Pretrained on the
100 million E-commerce-related data, ECLIP successfully extracts more generic,
semantic-rich, and robust representations. Extensive experimental results show
that, without further fine-tuning, ECLIP surpasses existing methods by a large
margin on a broad range of downstream tasks, demonstrating the strong
transferability to real-world E-commerce applications.

本研究提出了一种实例为中心的多模态预训练范式 ECLIP，通过引入可学习实例查询的解码器架构和两个预处理任务，从 1 亿个电子商务相关数据中预训练出语义丰富、稳健的模型，进而在广泛的下游任务中超越现有方法，证明其在现实世界中的广泛可转移性。