Active learning is the iterative construction of a classification model through targeted labeling, enabling significant labeling cost savings. As most research on active learning has been carried out before transformer-based language models ("transformers") became popular, despite its practical importance, comparably few papers have investigated how transformers can be combined with active learning to date. This can be attributed to the fact that using state-of-the-art query strategies for transformers induces a prohibitive runtime overhead, which effectively cancels out, or even outweighs aforementioned cost savings. In this paper, we revisit uncertainty-based query strategies, which had been largely outperformed before, but are particularly suited in the context of fine-tuning transformers. In an extensive evaluation on five widely used text classification benchmarks, we show that considerable improvements of up to 14.4 percentage points in area under the learning curve are achieved, as well as a final accuracy close to the state of the art for all but one benchmark, using only between 0.4% and 15% of the training data.

本文探讨了如何将基于不确定性的查询策略与transformers相结合以实现active learning，通过在五个广泛使用的文本分类基准测试中与其他不确定性策略进行比较，发现一些其他不确定性策略表现更好，挑战了prediction entropy查询策略在文本分类的active learning中最受欢迎的基线模型的地位。

基于不确定性的查询策略在使用Transformers进行主动学习中的再思考