In our increasingly interconnected world, where intelligent devices
continually amass copious personalized multi-modal data, a pressing need arises
to deliver high-quality, personalized device-aware services. However, this
endeavor presents a multifaceted challenge to prevailing artificial
intelligence (AI) systems primarily rooted in the cloud. As these systems
grapple with shifting data distributions between the cloud and devices, the
traditional approach of fine-tuning-based adaptation (FTA) exists the following
issues: the costly and time-consuming data annotation required by FTA and the
looming risk of model overfitting. To surmount these challenges, we introduce a
Universal On-Device Multi-modal Model Adaptation Framework, revolutionizing
on-device model adaptation by striking a balance between efficiency and
effectiveness. The framework features the Fast Domain Adaptor (FDA) hosted in
the cloud, providing tailored parameters for the Lightweight Multi-modal Model
on devices. To enhance adaptability across multi-modal tasks, the AnchorFrame
Distribution Reasoner (ADR) minimizes communication costs. Our contributions,
encapsulated in the Cloud-Device Collaboration Multi-modal Parameter Generation
(CDC-MMPG) framework, represent a pioneering solution for on-Device Multi-modal
Model Adaptation (DMMA). Extensive experiments validate the efficiency and
effectiveness of our method, particularly in video question answering and
retrieval tasks, driving forward the integration of intelligent devices into
our daily lives.

我们在日益相互连接的世界中，智能设备不断积累大量的个性化多模态数据，迫切需要提供高质量、个性化的设备感知服务。为了克服云端基于微调适应 (FTA) 存在的问题，我们引入了一种革新的通用 On-Device 多模态模型适应框架，通过在云端部署 Fast Domain Adaptor (FDA) 和设备上的轻量级多模态模型，实现了效率和效果的平衡。为了增强跨多模态任务的适应性，AnchorFrame Distribution Reasoner (ADR) 减少了通信成本。我们的贡献体现在 Cloud-Device Collaboration 多模态参数生成 (CDC-MMPG) 框架中，为 On-Device 多模态模型适应 (DMMA) 提供了先驱性的解决方案。广泛的实验证实了我们的方法的效率和效果，特别是在视频问答和检索任务中，推动了智能设备融入我们的日常生活。

无反向传播多模态在设备上通过云设备协作进行模型适应

Backpropogation-Free Multi-modal On-Device Model Adaptation via  Cloud-Device Collaboration

For various reasons including those related to climate changes, {\em energy}
has become a critical concern in all relevant activities and technical designs.
For the specific case of computer activities, the problem is exacerbated with
the emergence and pervasiveness of the so called {\em intelligent devices}.
From the application side, we point out the special topic of {\em Artificial
Intelligence}, who clearly needs an efficient computing support in order to
succeed in its purpose of being a {\em ubiquitous assistant}. There are mainly
two contexts where {\em energy} is one of the top priority concerns: {\em
embedded computing} and {\em supercomputing}. For the former, power consumption
is critical because the amount of energy that is available for the devices is
limited. For the latter, the heat dissipated is a serious source of failure and
the financial cost related to energy is likely to be a significant part of the
maintenance budget. On a single computer, the problem is commonly considered
through the electrical power consumption. This paper, written in the form of a
survey, we depict the landscape of energy concerns in computer activities, both
from the hardware and the software standpoints.

能源、智能设备、人工智能、嵌入式计算和超级计算是该研究论文中的关键词，并从硬件和软件的角度描述了计算活动中与能源相关的问题。

高性能计算系统和应用的能源问题

Energy Concerns with HPC Systems and Applications

Conversational understanding is an integral part of modern intelligent
devices. In a large fraction of the global traffic from customers using smart
digital assistants, frictions in dialogues may be attributed to incorrect
understanding of the entities in a customer's query due to factors including
ambiguous mentions, mispronunciation, background noise and faulty on-device
signal processing. Such errors are compounded by two common deficiencies from
intelligent devices namely, (1) the device not being tailored to individual
customers, and (2) the device responses being unaware of the context in the
conversation session. Viewing this problem via the lens of retrieval-based
search engines, we build and evaluate a scalable entity correction system,
PENTATRON. The system leverages a parametric transformer-based language model
to learn patterns from in-session customer-device interactions coupled with a
non-parametric personalized entity index to compute the correct query, which
aids downstream components in reasoning about the best response. In addition to
establishing baselines and demonstrating the value of personalized and
context-aware systems, we use multitasking to learn the domain of the correct
entity. We also investigate the utility of language model prompts. Through
extensive experiments, we show a significant upward movement of the key metric
(Exact Match) by up to 500.97% (relative to the baseline).

本文研究如何通过构建一种可扩展的实体矫正系统 PENTATRON，利用参数化的基于转换器的语言模型学习客户和设备之间的交互模式以及非参数化的个性化实体索引来计算正确的查询，从而帮助下游组件理解最佳响应，提高准确率。通过实验表明，该系统可以使关键指标（精确匹配）提高多达 500.97%。