Auto-labeling is an important family of techniques that produce labeled
training sets with minimum manual labeling. A prominent variant,
threshold-based auto-labeling (TBAL), works by finding a threshold on a model's
confidence scores above which it can accurately label unlabeled data points.
However, many models are known to produce overconfident scores, leading to poor
TBAL performance. While a natural idea is to apply off-the-shelf calibration
methods to alleviate the overconfidence issue, such methods still fall short.
Rather than experimenting with ad-hoc choices of confidence functions, we
propose a framework for studying the \emph{optimal} TBAL confidence function.
We develop a tractable version of the framework to obtain \texttt{Colander}
(Confidence functions for Efficient and Reliable Auto-labeling), a new post-hoc
method specifically designed to maximize performance in TBAL systems. We
perform an extensive empirical evaluation of our method \texttt{Colander} and
compare it against methods designed for calibration. \texttt{Colander} achieves
up to 60\% improvements on coverage over the baselines while maintaining
auto-labeling error below $5\%$ and using the same amount of labeled data as
the baselines.

使用阈值自动标记和最优置信度函数来提高自动标记系统的性能。方法 Colander 在基线模型的标签覆盖率上实现了高达 60% 的改进，并且保持自动标记错误率低于 5%，使用与基线相同数量的标记数据。