Numerous mobile apps have leveraged deep learning capabilities. However,
on-device models are vulnerable to attacks as they can be easily extracted from
their corresponding mobile apps. Existing on-device attacking approaches only
generate black-box attacks, which are far less effective and efficient than
white-box strategies. This is because mobile deep learning frameworks like
TFLite do not support gradient computing, which is necessary for white-box
attacking algorithms. Thus, we argue that existing findings may underestimate
the harmfulness of on-device attacks. To this end, we conduct a study to answer
this research question: Can on-device models be directly attacked via white-box
strategies? We first systematically analyze the difficulties of transforming
the on-device model to its debuggable version, and propose a Reverse
Engineering framework for On-device Models (REOM), which automatically reverses
the compiled on-device TFLite model to the debuggable model. Specifically, REOM
first transforms compiled on-device models into Open Neural Network Exchange
format, then removes the non-debuggable parts, and converts them to the
debuggable DL models format that allows attackers to exploit in a white-box
setting. Our experimental results show that our approach is effective in
achieving automated transformation among 244 TFLite models. Compared with
previous attacks using surrogate models, REOM enables attackers to achieve
higher attack success rates with a hundred times smaller attack perturbations.
In addition, because the ONNX platform has plenty of tools for model format
exchanging, the proposed method based on the ONNX platform can be adapted to
other model formats. Our findings emphasize the need for developers to
carefully consider their model deployment strategies, and use white-box methods
to evaluate the vulnerability of on-device models.

通过反向工程方法从移动设备应 用程序中彻底提取并逆向编译模型，研究了白盒攻击策略对移动设备模型的攻击效果，发现使用白盒策略能够大大提高攻击成功率并减小攻击干扰，强调了开发人员需要谨慎考虑模型部署策略，使用白盒方法评估移动设备模型的安全性。

研究针对设备端模型的白盒攻击

Investigating White-Box Attacks for On-Device Models

The requirements for many applications of state-of-the-art speech recognition
systems include not only low word error rate (WER) but also low latency.
Specifically, for many use-cases, the system must be able to decode utterances
in a streaming fashion and faster than real-time. Recently, a streaming
recurrent neural network transducer (RNN-T) end-to-end (E2E) model has shown to
be a good candidate for on-device speech recognition, with improved WER and
latency metrics compared to conventional on-device models [1]. However, this
model still lags behind a large state-of-the-art conventional model in quality
[2]. On the other hand, a non-streaming E2E Listen, Attend and Spell (LAS)
model has shown comparable quality to large conventional models [3]. This work
aims to bring the quality of an E2E streaming model closer to that of a
conventional system by incorporating a LAS network as a second-pass component,
while still abiding by latency constraints. Our proposed two-pass model
achieves a 17%-22% relative reduction in WER compared to RNN-T alone and
increases latency by a small fraction over RNN-T.

本研究旨在加入 Listen, Attend and Spell 模型作为第二步流程，从而将端到端流式模型的性能提高至与传统语音识别系统相当，同时也满足计算和响应延迟等约束条件。