BriefGPT.xyz
Oct, 2023
从CNN提炼高效的视觉Transformer用于语义分割
Distilling Efficient Vision Transformers from CNNs for Semantic Segmentation
HTML
PDF
Xu Zheng, Yunhao Luo, Pengyuan Zhou, Lin Wang
TL;DR
我们提出了一种CNN到ViT知识蒸馏框架,包括视觉语言特征蒸馏模块(VLFD)和像素级解耦蒸馏模块(PDD),实验证明我们的方法在三个语义分割基准数据集上的mIoU增量是最先进知识蒸馏方法的200%以上。
Abstract
In this paper, we tackle a new problem: how to transfer knowledge from the pre-trained cumbersome yet well-performed
cnn
-based model to learn a compact
vision transformer
(ViT)-based model while maintaining its l
→