This paper addresses spoken language understanding (SLU) on
microcontroller-like embedded devices, integrating on-device execution with
cloud offloading in a novel fashion. We exploit temporal locality in a device's
speech inputs and accordingly reuse recent SLU inferences. Our idea is simple:
let the device match new inputs against cached results, and only offload
unmatched inputs to the cloud for full inference. Realization of this idea,
however, is non-trivial: the device needs to compare acoustic features in a
robust, low-cost way. To this end, we present XYZ, a speech cache for tiny
devices. It matches speech inputs at two levels of representations: first by
clustered sequences of raw sound units, then as sequences of phonemes. Working
in tandem, the two representations offer complementary cost/accuracy tradeoffs.
To further boost accuracy, our cache is learning: with the mismatched and then
offloaded inputs, it continuously finetunes the device's feature extractors
(with the assistance of the cloud). We implement XYZ on an off-the-shelf STM32
microcontroller. The resultant implementation has a small memory footprint of
2MB. Evaluated on challenging speech benchmarks, our system resolves 45%--90%
of inputs on device, reducing the average latency by up to 80% compared to
offloading to popular cloud speech services. Our benefit is pronounced even in
adversarial settings -- noisy environments, cold cache, or one device shared by
a number of users.

探讨在类微控制器嵌入式设备上的口语理解（SLU），将设备上的执行与云端卸载结合在一起。通过在设备的语音输入中利用时间局部性，重复使用最近的 SLU 推断。我们的想法很简单：让设备将新输入与缓存结果进行匹配，并只将匹配不成功的输入卸载到云端进行完整的推断。实现这个想法，然而，是非常复杂的：设备需要以稳健、低成本的方式比较声学特征。为了达到这个目的，我们提出了 XYZ，一种适用于微型设备的语音缓存。它通过聚类的原始声音单元序列和音素序列两个级别来匹配语音输入，两个表示方法相辅相成，提供了成本 / 准确性的互补权衡。为了进一步提高准确性，我们的缓存是可以学习的：通过不匹配的以及卸载到云端的输入，它不断地微调设备的特征提取器（在云端的辅助下）。我们在一款现成的 STM32 微控制器上实现了 XYZ。实现的结果具有 2MB 的小内存占用量。在具有挑战性的语音基准测试中进行评估时，我们的系统能够在设备上解决 45% 至 90% 的输入，与卸载到流行的云语音服务相比，平均延迟降低了高达 80%。即使在对抗性环境（噪声环境、冷缓存或多个用户共享设备）中，我们的优势仍然显著。

利用缓存在小型设备上实现 SLU

Leveraging cache to enable SLU on tiny devices

Image recognition is an essential task in several mobile applications. For
instance, a smartphone can process a landmark photo to gather more information
about its location. If the device does not have enough computational resources
available, it offloads the processing task to a cloud infrastructure. Although
this approach solves resource shortages, it introduces a communication delay.
Image-recognition caches on the Internet's edge can mitigate this problem.
These caches run on servers close to mobile devices and stores information
about previously recognized images. If the server receives a request with a
photo stored in its cache, it replies to the device, avoiding cloud offloading.
The main challenge for this cache is to verify if the received image matches a
stored one. Furthermore, for outdoor photos, it is difficult to compare them if
one was taken in the daytime and the other at nighttime. In that case, the
cache might wrongly infer that they refer to different places, offloading the
processing to the cloud. This work shows that a well-known generative
adversarial network, called ToDayGAN, can solve this problem by generating
daytime images using nighttime ones. We can thus use this translation to
populate a cache with synthetic photos that can help image matching. We show
that our solution reduces cloud offloading and, therefore, the application's
latency.

本研究介绍使用 ToDayGAN 生成夜景照片的白天照片来解决云端处理通信延迟的问题，并通过在 Internet 边缘缓存运行的图像识别缓存验证其有效性。