With the success of large-scale visual-language pretraining models and the
wide application of image-text retrieval in industry areas, reducing the model
size and streamlining their terminal-device deployment have become urgently
necessary. The mainstream model structures for image-text retrieval are
single-stream and dual-stream, both aiming to close the semantic gap between
visual and textual modalities. Dual-stream models excel at offline indexing and
fast inference, while single-stream models achieve more accurate cross-model
alignment by employing adequate feature fusion. We propose a multi-teacher
cross-modality alignment distillation (MCAD) technique to integrate the
advantages of single-stream and dual-stream models. By incorporating the fused
single-stream features into the image and text features of the dual-stream
model, we formulate new modified teacher features and logits. Then, we conduct
both logit and feature distillation to boost the capability of the student
dual-stream model, achieving high retrieval performance without increasing
inference complexity. Extensive experiments demonstrate the remarkable
performance and high efficiency of MCAD on image-text retrieval tasks.
Furthermore, we implement a mobile CLIP model on Snapdragon clips with only 93M
running memory and 30ms search latency, without apparent performance
degradation of the original large CLIP.

使用多教师跨模态对齐蒸馏技术 (MCAD)，通过在双流模型中融合单流特征提高学生模型的检索性能，同时实现高效的图像 - 文本检索任务，降低模型大小和终端设备部署的复杂性。