Deep Neural Networks (DNNs) are extremely computationally demanding, which
presents a large barrier to their deployment on resource-constrained devices.
Since such devices are where many emerging deep learning applications lie
(e.g., drones, vision-based medical technology), significant bodies of work
from both the machine learning and systems communities have attempted to
provide optimizations to accelerate DNNs. To help unify these two perspectives,
in this paper we combine machine learning and systems techniques within the
Deep Learning Acceleration Stack (DLAS), and demonstrate how these layers can
be tightly dependent on each other with an across-stack perturbation study. We
evaluate the impact on accuracy and inference time when varying different
parameters of DLAS across two datasets, seven popular DNN architectures, four
DNN compression techniques, three algorithmic primitives with sparse and dense
variants, untuned and auto-scheduled code generation, and four hardware
platforms. Our evaluation highlights how perturbations across DLAS parameters
can cause significant variation and across-stack interactions. The highest
level observation from our evaluation is that the model size, accuracy, and
inference time are not guaranteed to be correlated. Overall we make 13 key
observations, including that speedups provided by compression techniques are
very hardware dependent, and that compiler auto-tuning can significantly alter
what the best algorithm to use for a given configuration is. With DLAS, we aim
to provide a reference framework to aid machine learning and systems
practitioners in reasoning about the context in which their respective DNN
acceleration solutions exist in. With our evaluation strongly motivating the
need for co-design, we believe that DLAS can be a valuable concept for
exploring the next generation of co-designed accelerated deep learning
solutions.

深度神经网络的加速和优化在资源有限的设备上是具有挑战性的，这篇论文结合了机器学习和系统技术，在 Deep Learning Acceleration Stack (DLAS) 中演示了如何通过整合不同层面的改进技术来提高加速效果，并强调了由于参数调整引起的 DLAS 各层之间的相互影响。研究发现，模型大小、准确性和推理时间之间并没有必然的关联，压缩技术的加速效果也受具体硬件平台的影响，而编译器自动调优可能会改变最佳算法的选择，因此需要进行协同设计来进一步优化加速深度学习的解决方案。

DLAS：深度学习加速器方案的探索与评估

DLAS: An Exploration and Assessment of the Deep Learning Acceleration  Stack

We have designed, developed, and applied a convolutional neural network (CNN)
architecture using multi-task learning to search for and characterize strong HI
Lya absorption in quasar spectra. Without any explicit modeling of the quasar
continuum nor application of the predicted line-profile for Lya from quantum
mechanics, our algorithm predicts the presence of strong HI absorption and
estimates the corresponding redshift zabs and HI column density NHI, with
emphasis on damped Lya systems (DLAs, absorbers with log NHI > 20.3). We tuned
the CNN model using a custom training set of DLAs injected into DLA-free quasar
spectra from the Sloan Digital Sky Survey (SDSS), data release 5 (DR5). Testing
on a held-back validation set demonstrates a high incidence of DLAs recovered
by the algorithm (97.4% as DLAs and 99% as an HI absorber with log NHI > 19.5)
and excellent estimates for zabs and NHI. Similar results are obtained against
a human-generated survey of the SDSS DR5 dataset. The algorithm yields a low
incidence of false positives and negatives but is challenged by overlapping
DLAs and/or very high NHI systems. We have applied this CNN model to the quasar
spectra of SDSS-DR7 and the Baryonic Oscillation Spectroscopic Survey (BOSS,
data release 12) and provide catalogs of 4,913 and 50,969 DLAs respectively
(including 1,659 and 9,230 high-confidence DLAs that were previously
unpublished). This work validates the application of deep learning techniques
to astronomical spectra for both classification and quantitative measurements.

本篇论文使用多目标学习的卷积神经网络来搜寻和表征夸克星光谱中的强氢 Lyman-Alpha 吸收，并完成分类和定量测量。该算法能够准确地预测 HI 吸收的存在性，并估计相应的 redshift zabs 和 HI 柱密度 NHI，并使用自定义的训练集进行模型训练，最后在 SDSS-DR7 和 BOSS 数据集中应用，证明了深度学习在天文光谱测量方面应用的有效性。