Transformer language models provide superior accuracy over previous models
but they are computationally and environmentally expensive. Borrowing the
concept of model cascading from computer vision, we introduce BabyBear, a
framework for cascading models for natural language processing (NLP) tasks to
minimize cost. The core strategy is inference triage, exiting early when the
least expensive model in the cascade achieves a sufficiently high-confidence
prediction. We test BabyBear on several open source data sets related to
document classification and entity recognition. We find that for common NLP
tasks a high proportion of the inference load can be accomplished with cheap,
fast models that have learned by observing a deep learning model. This allows
us to reduce the compute cost of large-scale classification jobs by more than
50% while retaining overall accuracy. For named entity recognition, we save 33%
of the deep learning compute while maintaining an F1 score higher than 95% on
the CoNLL benchmark.

BabyBear 是一种自然语言处理的 CASCADING MODEL 框架，其中采用了 Inference Triage 策略，使得在分类等任务中，减少了计算成本，同时仍然可以达到较高精度。

BabyBear：针对昂贵语言模型的廉价推理分流

BabyBear: Cheap inference triage for expensive language models

Over its three decade history, speech translation has experienced several
shifts in its primary research themes; moving from loosely coupled cascades of
speech recognition and machine translation, to exploring questions of tight
coupling, and finally to end-to-end models that have recently attracted much
attention. This paper provides a brief survey of these developments, along with
a discussion of the main challenges of traditional approaches which stem from
committing to intermediate representations from the speech recognizer, and from
training cascaded models separately towards different objectives.
Recent end-to-end modeling techniques promise a principled way of overcoming
these issues by allowing joint training of all model components and removing
the need for explicit intermediate representations. However, a closer look
reveals that many end-to-end models fall short of solving these issues, due to
compromises made to address data scarcity. This paper provides a unifying
categorization and nomenclature that covers both traditional and recent
approaches and that may help researchers by highlighting both trade-offs and
open research questions.

本文介绍了语音翻译技术从松散耦合的串联模型到紧耦合再到最近备受关注的端到端模型的发展历程，讨论了传统方法所面临的主要挑战和最新方法对克服这些问题的方案。通过对传统和最新方法进行统一分类和命名，以解决语音识别和数据缺失问题等开放性研究问题。