An active learner is given a hypothesis class, a large set of unlabeled
examples and the ability to interactively query labels to an oracle of a subset
of these examples; the goal of the learner is to learn a hypothesis in the
class that fits the data well by making as few label queries as possible.
This work addresses active learning with labels obtained from strong and weak
labelers, where in addition to the standard active learning setting, we have an
extra weak labeler which may occasionally provide incorrect labels. An example
is learning to classify medical images where either expensive labels may be
obtained from a physician (oracle or strong labeler), or cheaper but
occasionally incorrect labels may be obtained from a medical resident (weak
labeler). Our goal is to learn a classifier with low error on data labeled by
the oracle, while using the weak labeler to reduce the number of label queries
made to this labeler. We provide an active learning algorithm for this setting,
establish its statistical consistency, and analyze its label complexity to
characterize when it can provide label savings over using the strong labeler
alone.

本文探讨了一种在拥有弱标注者和强标注者存在的情况下进行主动学习的算法，旨在学习一个在正确标注数据上误差较小的分类器，同时通过弱标注者减少标注者的查询次数，从而降低标注的成本。我们提出了一种主动学习算法，并建立了其统计一致性，分析了其标注复杂性，以便表明是否能够在使用强标注者的基础上节约标注成本。