BriefGPT.xyz
Dec, 2023
FerKD:高效蒸馏的手术标记适应
FerKD: Surgical Label Adaptation for Efficient Distillation
HTML
PDF
Zhiqiang Shen
TL;DR
FerKD是一种新的高效知识蒸馏框架,结合了部分软硬标签自适应和区域校准机制;采用自校正的软化逼近于真实标签的较不可信可信区域机制作为背景,显著提高了收敛速度和最终准确性,并在ImageNet-1K和下游任务中取得显著改进。
Abstract
We present
ferkd
, a novel efficient
knowledge distillation
framework that incorporates partial
soft-hard label adaptation
coupled with a <
→