BriefGPT.xyz
Apr, 2024
DeiT-LT蒸馏对长尾数据集的视觉Transformer训练再次起效
DeiT-LT Distillation Strikes Back for Vision Transformer Training on Long-Tailed Datasets
HTML
PDF
Harsh Rangwani, Pradipto Mondal, Mayank Mishra, Ashish Ramayee Asokan, R. Venkatesh Babu
TL;DR
利用DeiT-LT方案能够在长尾数据集上从头开始训练ViT,并使用蒸馏DIST令牌通过在ViT架构中使用不同的令牌来学习对应于多数和少数类的特征。
Abstract
vision transformer
(
vit
) has emerged as a prominent architecture for various computer vision tasks. In
vit
, we divide the input image into
→