With the emergence of deep learning techniques, smartphone apps are now
embedded on-device AI features for enabling advanced tasks like speech
translation, to attract users and increase market competitiveness. A good
interaction design is important to make an AI feature usable and
understandable. However, AI features have their unique challenges like
sensitiveness to the input, dynamic behaviours and output uncertainty. Existing
guidelines and tools either do not cover AI features or consider mobile apps
which are confirmed by our informal interview with professional designers. To
address these issues, we conducted the first empirical study to explore
user-AI-interaction in mobile apps. We aim to understand the status of
on-device AI usage by investigating 176 AI apps from 62,822 apps. We identified
255 AI features and summarised 759 implementations into three primary
interaction pattern types. We further implemented our findings into a
multi-faceted search-enabled gallery. The results of the user study demonstrate
the usefulness of our findings.

本研究探索手机应用程序中的人工智能功能使用现状，总结了三种主要的交互模式类型，并实施了一个多方面的搜索启用画廊，研究结果表明了我们发现的有用性。

朝着真正的智能应用程序：研究智能手机内置 AI 应用程序中的人工智能交互

Towards Real Smart Apps: Investigating Human-AI Interactions in  Smartphone On-Device AI Apps

Designing suitable deep model architectures, for AI-driven on-device apps and
features, at par with rapidly evolving mobile hardware and increasingly complex
target scenarios is a difficult task. Though Neural Architecture Search
(NAS/AutoML) has made this easier by shifting paradigm from extensive manual
effort to automated architecture learning from data, yet it has major
limitations, leading to critical bottlenecks in the context of mobile devices,
including model-hardware fidelity, prohibitive search times and deviation from
primary target objective(s). Thus, we propose AutoCoMet that can learn the most
suitable DNN architecture optimized for varied types of device hardware and
task contexts, ~ 3x faster. Our novel co-regulated shaping reinforcement
controller together with the high fidelity hardware meta-behavior predictor
produces a smart, fast NAS framework that adapts to context via a generalized
formalism for any kind of multi-criteria optimization.

AutoCoMet 是一种智能、快速的神经架构搜索框架，利用协同规范形成的强化控制器和高保真硬件元行为预测模型，通过一般化的多准则优化形式适应各种设备硬件和任务上下文，使得深度模型架构优化比传统手动设计方法快 3 倍左右。

AutoCoMet: 基于协同调节塑形增强的智能神经结构搜索

AutoCoMet: Smart Neural Architecture Search via Co-Regulated Shaping Reinforcement

We propose nnstreamer, a software system that handles neural networks as
filters of stream pipelines, applying the stream processing paradigm to neural
network applications. A new trend with the wide-spread of deep neural network
applications is on-device AI; i.e., processing neural networks directly on
mobile devices or edge/IoT devices instead of cloud servers. Emerging privacy
issues, data transmission costs, and operational costs signifies the need for
on-device AI especially when a huge number of devices with real-time data
processing are deployed. Nnstreamer efficiently handles neural networks with
complex data stream pipelines on devices, improving the overall performance
significantly with minimal efforts. Besides, nnstreamer simplifies the neural
network pipeline implementations and allows reusing off-shelf multimedia stream
filters directly; thus it reduces the developmental costs significantly.
Nnstreamer is already being deployed with a product releasing soon and is open
source software applicable to a wide range of hardware architectures and
software platforms.

nnstreamer 是一种以流处理范式为基础，将神经网络作为流水线过滤器处理的软件系统，可以高效地处理复杂的数据流模型，降低开发成本，实现在设备端进行 AI 处理。