Pre-trained language models have shown stellar performance in various
downstream tasks. But, this usually comes at the cost of high latency and
computation, hindering their usage in resource-limited settings. In this work,
we propose a novel approach for reducing the computational cost of BERT with
minimal loss in downstream performance. Our method dynamically eliminates less
contributing tokens through layers, resulting in shorter lengths and
consequently lower computational cost. To determine the importance of each
token representation, we train a Contribution Predictor for each layer using a
gradient-based saliency method. Our experiments on several diverse
classification tasks show speedups up to 22x during inference time without much
sacrifice in performance. We also validate the quality of the selected tokens
in our method using human annotations in the ERASER benchmark. In comparison to
other widely used strategies for selecting important tokens, such as saliency
and attention, our proposed method has a significantly lower false positive
rate in generating rationales. Our code is freely available at
this https URL .

本文提出了一种基于 salience 方法的计算成本降低技术，具体来说，是通过动态消除 BERT 模型中的无用 tokens 以降低计算成本，在几个不同的分类任务中验证其性能，实现了高达 22 倍的加速。

AdapLeR: 自适应长度缩减加速推理

AdapLeR: Speeding up Inference by Adaptive Length Reduction

This paper introduces Dex, a reinforcement learning environment toolkit
specialized for training and evaluation of continual learning methods as well
as general reinforcement learning problems. We also present the novel continual
learning method of incremental learning, where a challenging environment is
solved using optimal weight initialization learned from first solving a similar
easier environment. We show that incremental learning can produce vastly
superior results than standard methods by providing a strong baseline method
across ten Dex environments. We finally develop a saliency method for
qualitative analysis of reinforcement learning, which shows the impact
incremental learning has on network attention.

介绍了针对连续学习方法培训和评估的 DEX 强化学习环境工具包以及增量学习方法的巧妙应用，该方法可以通过从解决类似的较简单的环境中学习的最佳重量初始化来解决挑战性环境，并通过提供十种 DEX 环境的强大基线方法表明增量学习可以产生远远优于标准方法的结果，并最终开发了一种用于定性分析强化学习的显著性方法，展示了增量学习对网络注意力的影响。