We introduce OmniSource, a novel framework for leveraging web data to train
video recognition models. OmniSource overcomes the barriers between data
formats, such as images, short videos, and long untrimmed videos for
webly-supervised learning. First, data samples with multiple formats, curated
by task-specific data collection and automatically filtered by a teacher model,
are transformed into a unified form. Then a joint-training strategy is proposed
to deal with the domain gaps between multiple data sources and formats in
webly-supervised learning. Several good practices, including data balancing,
resampling, and cross-dataset mixup are adopted in joint training. Experiments
show that by utilizing data from multiple sources and formats, OmniSource is
more data-efficient in training. With only 3.5M images and 800K minutes videos
crawled from the internet without human labeling (less than 2% of prior works),
our models learned with OmniSource improve Top-1 accuracy of 2D- and 3D-ConvNet
baseline models by 3.0% and 3.9%, respectively, on the Kinetics-400 benchmark.
With OmniSource, we establish new records with different pretraining strategies
for video recognition. Our best models achieve 80.4%, 80.5%, and 83.6 Top-1
accuracies on the Kinetics-400 benchmark respectively for
training-from-scratch, ImageNet pre-training and IG-65M pre-training.

OmniSource 是一个使用多种数据源和格式进行联合训练的框架，能够在没有人类标记的情况下从网上抓取 3.5M 的图片和 800K 分钟的视频，提高了在视频识别领域的表现。