Recent advancements in Vision-Language (VL) models have sparked interest in
their deployment on edge devices, yet challenges in handling diverse visual
modalities, manual annotation, and computational constraints remain. We
introduce EdgeVL, a novel framework that bridges this gap by seamlessly
integrating dual-modality knowledge distillation and quantization-aware
contrastive learning. This approach enables the adaptation of large VL models,
like CLIP, for efficient use with both RGB and non-RGB images on
resource-limited devices without the need for manual annotations. EdgeVL not
only transfers visual language alignment capabilities to compact models but
also maintains feature quality post-quantization, significantly enhancing
open-vocabulary classification performance across various visual modalities.
Our work represents the first systematic effort to adapt large VL models for
edge deployment, showcasing up to 15.4% accuracy improvements on multiple
datasets and up to 93-fold reduction in model size.

通过双模态知识蒸馏和量化感知对比学习，EdgeVL 框架实现了大型视觉语言模型在资源受限设备上的高效使用，无需手动标注，提高了在多种视觉模态下的开放词汇分类性能，同时在多个数据集上提升了 15.4% 的准确性，并在模型规模上缩小了 93 倍。