BriefGPT.xyz
Dec, 2022
具有上下文目标表示的视觉、语音和语言自监督高效学习
Efficient Self-supervised Learning with Contextualized Target Representations for Vision, Speech and Language
HTML
PDF
Alexei Baevski, Arun Babu, Wei-Ning Hsu, Michael Auli
TL;DR
本文提出 data2vec 2.0 算法,通过利用丰富的上下文目标表示,实现了在几个模态之间进行泛化的快速自监督学习,进而在图像分类、语音识别等领域取得了很好的实验效果。
Abstract
Current
self-supervised learning
algorithms are often modality-specific and require large amounts of computational resources. To address these issues, we increase the training efficiency of
data2vec
, a learning o
→