We introduce OmniSource, a novel framework for leveraging web data to train
video recognition models. OmniSource overcomes the barriers between data
formats, such as images, short videos, and long untrimmed videos for
webly-supervised learning. First, data samples with multiple formats, curated
by task-specific data collection and automatically filtered by a teacher model,
are transformed into a unified form. Then a joint-training strategy is proposed
to deal with the domain gaps between multiple data sources and formats in
webly-supervised learning. Several good practices, including data balancing,
resampling, and cross-dataset mixup are adopted in joint training. Experiments
show that by utilizing data from multiple sources and formats, OmniSource is
more data-efficient in training. With only 3.5M images and 800K minutes videos
crawled from the internet without human labeling (less than 2% of prior works),
our models learned with OmniSource improve Top-1 accuracy of 2D- and 3D-ConvNet
baseline models by 3.0% and 3.9%, respectively, on the Kinetics-400 benchmark.
With OmniSource, we establish new records with different pretraining strategies
for video recognition. Our best models achieve 80.4%, 80.5%, and 83.6 Top-1
accuracies on the Kinetics-400 benchmark respectively for
training-from-scratch, ImageNet pre-training and IG-65M pre-training.

OmniSource 是一个使用多种数据源和格式进行联合训练的框架，能够在没有人类标记的情况下从网上抓取 3.5M 的图片和 800K 分钟的视频，提高了在视频识别领域的表现。

全源网络监督学习用于视频识别

Omni-sourced Webly-supervised Learning for Video Recognition

Understanding the simultaneously very diverse and intricately fine-grained
set of possible human actions is a critical open problem in computer vision.
Manually labeling training videos is feasible for some action classes but
doesn't scale to the full long-tailed distribution of actions. A promising way
to address this is to leverage noisy data from web queries to learn new
actions, using semi-supervised or "webly-supervised" approaches. However, these
methods typically do not learn domain-specific knowledge, or rely on iterative
hand-tuned data labeling policies. In this work, we instead propose a
reinforcement learning-based formulation for selecting the right examples for
training a classifier from noisy web search results. Our method uses Q-learning
to learn a data labeling policy on a small labeled training dataset, and then
uses this to automatically label noisy web data for new visual concepts.
Experiments on the challenging Sports-1M action recognition benchmark as well
as on additional fine-grained and newly emerging action classes demonstrate
that our method is able to learn good labeling policies for noisy data and use
this to learn accurate visual concept classifiers.

本文提出了基于强化学习的标记策略，从嘈杂的网络搜索结果中选择正确的样本来训练分类器，以学习准确的视觉概念分类器。实验结果表明，我们的方法能够学习嘈杂数据的良好标记策略，并用此学习精确的视觉概念分类器。