Self-supervised speech models have shown to be useful for various tasks, but
their large size limits the use in devices with low computing power and memory.
In this work, we explore early exit, an approach for reducing latency by
exiting the forward process of a network early. Most approaches of early exit
need a separate early exit model for each task, with some even requiring
fine-tuning of the entire pretrained model. We introduce Data Adaptive
Self-Supervised Early Exit (DAISY), an approach that decides when to exit based
on the self-supervised loss, eliminating the need for multiple round of
training and fine-tuning. DAISY matches the performance of HuBERT on the
MiniSUPERB benchmark, but with much faster inference times. Our analysis on the
adaptivity of DAISY shows that the model exits early (using fewer layers) on
clean data while exits late (using more layers) on noisy data, dynamically
adjusting the computational cost of inference based on the noise level of each
sample.

自主训练的语音模型在各种任务中已显示出其有用性，但其庞大的体积限制了在计算能力和内存较低的设备中的使用。本论文探讨了一种早期退出的方法，用于通过尽早退出网络的前向过程来减少延迟。我们介绍了数据自适应自主训练早期退出（DAISY）方法，该方法根据自主训练损失来决定何时退出，消除了多轮训练和微调的需求。DAISY 在 MiniSUPERB 基准测试上与 HuBERT 的性能相匹配，但推理时间更快。我们对 DAISY 的适应性进行了分析，结果显示该模型在干净数据上早期退出（使用较少层），而在噪声数据上晚期退出（使用更多层），根据每个样本的噪声水平动态调整推理的计算成本。

DAISY: 自适应数据自我监督的语音表示模型的早期退出

DAISY: Data Adaptive Self-Supervised Early Exit for Speech  Representation Models

Traditional language models operate autoregressively, i.e., they predict one
token at a time. Rapid explosion in model sizes has resulted in high inference
times. In this work, we propose DynaMo, a suite of multi-token prediction
language models that reduce net inference times. Our models
$\textit{dynamically}$ predict multiple tokens based on their confidence in the
predicted joint probability distribution. We propose a lightweight technique to
train these models, leveraging the weights of traditional autoregressive
counterparts. Moreover, we propose novel ways to enhance the estimated joint
probability to improve text generation quality, namely co-occurrence weighted
masking and adaptive thresholding. We also propose systematic qualitative and
quantitative methods to rigorously test the quality of generated text for
non-autoregressive generation. One of the models in our suite, DynaMo-7.3B-T3,
achieves same-quality generated text as the baseline (Pythia-6.9B) while
achieving 2.57$\times$ speed-up with only 5.87% and 2.67% parameter and
training time overheads, respectively.

DynaMo 是一个多令牌预测语言模型套件，通过在预测联合概率分布的基础上动态预测多个令牌来减少净推理时间，实现了与基准 (Pythia-6.9B) 相同质量的生成文本，并获得 2.57 倍的加速，仅有 5.87% 和 2.67% 的参数和训练时间开销。

DynaMo: 动态多词采样加速语言模型推断

DynaMo: Accelerating Language Model Inference with Dynamic Multi-Token  Sampling

We present a real-time method for robust estimation of multiple instances of
geometric models from noisy data. Geometric models such as vanishing points,
planar homographies or fundamental matrices are essential for 3D scene
analysis. Previous approaches discover distinct model instances in an iterative
manner, thus limiting their potential for speedup via parallel computation. In
contrast, our method detects all model instances independently and in parallel.
A neural network segments the input data into clusters representing potential
model instances by predicting multiple sets of sample and inlier weights. Using
the predicted weights, we determine the model parameters for each potential
instance separately in a RANSAC-like fashion. We train the neural network via
task-specific loss functions, i.e. we do not require a ground-truth
segmentation of the input data. As suitable training data for homography and
fundamental matrix fitting is scarce, we additionally present two new synthetic
datasets. We demonstrate state-of-the-art performance on these as well as
multiple established datasets, with inference times as small as five
milliseconds per image.

我们提出了一种实时方法，用于从嘈杂数据中稳健地估计多个几何模型实例。

PARSAC: 并行样本一致性加速健壮的多模型拟合

PARSAC: Accelerating Robust Multi-Model Fitting with Parallel Sample  Consensus

Recent research has explored the implementation of privacy-preserving deep
neural networks solely using fully homomorphic encryption. However, its
practicality has been limited because of prolonged inference times. When using
a pre-trained model without retraining, a major factor contributing to these
prolonged inference times is the high-degree polynomial approximation of
activation functions such as the ReLU function. The high-degree approximation
consumes a substantial amount of homomorphic computational resources, resulting
in slower inference. Unlike the previous works approximating activation
functions uniformly and conservatively, this paper presents a \emph{layerwise}
degree optimization of activation functions to aggressively reduce the
inference time while maintaining classification accuracy by taking into account
the characteristics of each layer. Instead of the minimax approximation
commonly used in state-of-the-art private inference models, we employ the
weighted least squares approximation method with the input distributions of
activation functions. Then, we obtain the layerwise optimized degrees for
activation functions through the \emph{dynamic programming} algorithm,
considering how each layer's approximation error affects the classification
accuracy of the deep neural network. Furthermore, we propose modulating the
ciphertext moduli-chain layerwise to reduce the inference time. By these
proposed layerwise optimization methods, we can reduce inference times for the
ResNet-20 model and the ResNet-32 model by 3.44 times and 3.16 times,
respectively, in comparison to the prior implementations employing uniform
degree polynomials and a consistent ciphertext modulus.

通过对激活函数进行逐层优化并调整密文模量链，本文提出了一种降低推理时间的隐私保护深度神经网络实现方法，可以将 ResNet-20 模型和 ResNet-32 模型的推理时间分别缩短 3.44 倍和 3.16 倍。