The security guarantee of AI-enabled software systems (particularly using
deep learning techniques as a functional core) is pivotal against the
adversarial attacks exploiting software vulnerabilities. However, little
attention has been paid to a systematic investigation of vulnerabilities in
such systems. A common situation learned from the open source software
community is that deep learning engineers frequently integrate off-the-shelf or
open-source learning frameworks into their ecosystems. In this work, we
specifically look into deep learning (DL) framework and perform the first
systematic study of vulnerabilities in DL systems through a comprehensive
analysis of identified vulnerabilities from Common Vulnerabilities and
Exposures (CVE) and open-source DL tools, including TensorFlow, Caffe, OpenCV,
Keras, and PyTorch. We propose a two-stream data analysis framework to explore
vulnerability patterns from various databases. We investigate the unique DL
frameworks and libraries development ecosystems that appear to be decentralized
and fragmented. By revisiting the Common Weakness Enumeration (CWE) List, which
provides the traditional software vulnerability related practices, we observed
that it is more challenging to detect and fix the vulnerabilities throughout
the DL systems lifecycle. Moreover, we conducted a large-scale empirical study
of 3,049 DL vulnerabilities to better understand the patterns of vulnerability
and the challenges in fixing them. We have released the full replication
package at this https URL We anticipate
that our study can advance the development of secure DL systems.

通过对 AI 软件系统中深度学习技术的脆弱性进行系统研究，我们提出了一个两流数据分析框架，探索各种数据库中的脆弱性模式，并进行了大规模实证研究来理解脆弱性的模式和修复中的挑战，以推进安全的深度学习系统的发展。

深度学习系统的安全弱点和漏洞

On Security Weaknesses and Vulnerabilities in Deep Learning Systems

Deep learning (DL) frameworks take advantage of GPUs to improve the speed of
DL inference and training. Ideally, DL frameworks should be able to fully
utilize the computation power of GPUs such that the running time depends on the
amount of computation assigned to GPUs. Yet, we observe that in scheduling GPU
tasks, existing DL frameworks suffer from inefficiencies such as large
scheduling overhead and unnecessary serial execution. To this end, we propose
Nimble, a DL execution engine that runs GPU tasks in parallel with minimal
scheduling overhead. Nimble introduces a novel technique called ahead-of-time
(AoT) scheduling. Here, the scheduling procedure finishes before executing the
GPU kernel, thereby removing most of the scheduling overhead during run time.
Furthermore, Nimble automatically parallelizes the execution of GPU tasks by
exploiting multiple GPU streams in a single GPU. Evaluation on a variety of
neural networks shows that compared to PyTorch, Nimble speeds up inference and
training by up to 22.34$\times$ and 3.61$\times$, respectively. Moreover,
Nimble outperforms state-of-the-art inference systems, TensorRT and TVM, by up
to 2.81$\times$ and 1.70$\times$, respectively.

本文介绍了一种深度学习执行引擎 Nimble，采用 ahead-of-time（AoT）调度技术，可以以尽可能小的调度开销运行 GPU 任务。与 PyTorch, TensorRT 和 TVM 相比，Nimble 在加速推断和训练方面表现出较好的性能。

Nimble: 适用于深度学习的轻量级和并行 GPU 任务调度

Nimble: Lightweight and Parallel GPU Task Scheduling for Deep Learning

Deep Learning (DL) has recently achieved tremendous success. A variety of DL
frameworks and platforms play a key role to catalyze such progress. However,
the differences in architecture designs and implementations of existing
frameworks and platforms bring new challenges for DL software development and
deployment. Till now, there is no study on how various mainstream frameworks
and platforms influence both DL software development and deployment in
practice. To fill this gap, we take the first step towards understanding how
the most widely-used DL frameworks and platforms support the DL software
development and deployment. We conduct a systematic study on these frameworks
and platforms by using two types of DNN architectures and three popular
datasets. (1) For development process, we investigate the prediction accuracy
under the same runtime training configuration or same model weights/biases. We
also study the adversarial robustness of trained models by leveraging the
existing adversarial attack techniques. The experimental results show that the
computing differences across frameworks could result in an obvious prediction
accuracy decline, which should draw the attention of DL developers. (2) For
deployment process, we investigate the prediction accuracy and performance
(refers to time cost and memory consumption) when the trained models are
migrated/quantized from PC to real mobile devices and web browsers. The DL
platform study unveils that the migration and quantization still suffer from
compatibility and reliability issues. Meanwhile, we find several DL software
bugs by using the results as a benchmark. We further validate the results
through bug confirmation from stakeholders and industrial positive feedback to
highlight the implications of our study. Through our study, we summarize
practical guidelines, identify challenges and pinpoint new research directions.

该研究对主流深度学习框架和平台在开发和部署中的影响进行了系统研究，结果表明计算差异可能导致项目的预测准确度下降，部署过程仍存在兼容性和可靠性问题，并通过发现的问题总结出实用的指南以及明确新的研究方向。

一个旨在表征不同框架和平台下深度学习开发与部署的实证研究

An Empirical Study towards Characterizing Deep Learning Development and  Deployment across Different Frameworks and Platforms

Homomorphic encryption (HE)---the ability to perform computation on encrypted
data---is an attractive remedy to increasing concerns about data privacy in
deep learning (DL). However, building DL models that operate on ciphertext is
currently labor-intensive and requires simultaneous expertise in DL,
cryptography, and software engineering. DL frameworks and recent advances in
graph compilers have greatly accelerated the training and deployment of DL
models to various computing platforms. We introduce nGraph-HE, an extension of
nGraph, Intel's DL graph compiler, which enables deployment of trained models
with popular frameworks such as TensorFlow while simply treating HE as another
hardware target. Our graph-compiler approach enables HE-aware optimizations--
implemented at compile-time, such as constant folding and HE-SIMD packing, and
at run-time, such as special value plaintext bypass. Furthermore, nGraph-HE
integrates with DL frameworks such as TensorFlow, enabling data scientists to
benchmark DL models with minimal overhead.

Intel's nGraph-HE, an extension of nGraph, enables the deployment of deep learning models on ciphertext using homomorphic encryption as a hardware target, allowing DL-aware optimizations and integration with popular frameworks such as TensorFlow to benchmark models with minimal overhead.