The deployment of Large Language Models (LLMs) and Large Multimodal Models
(LMMs) on mobile devices has gained significant attention due to the benefits
of enhanced privacy, stability, and personalization. However, the hardware
constraints of mobile devices necessitate the use of models with fewer
parameters and model compression techniques like quantization. Currently, there
is limited understanding of quantization's impact on various task performances,
including LLM tasks, LMM tasks, and, critically, trust and safety. There is a
lack of adequate tools for systematically testing these models on mobile
devices. To address these gaps, we introduce MobileAIBench, a comprehensive
benchmarking framework for evaluating mobile-optimized LLMs and LMMs.
MobileAIBench assesses models across different sizes, quantization levels, and
tasks, measuring latency and resource consumption on real devices. Our two-part
open-source framework includes a library for running evaluations on desktops
and an iOS app for on-device latency and hardware utilization measurements. Our
thorough analysis aims to accelerate mobile AI research and deployment by
providing insights into the performance and feasibility of deploying LLMs and
LMMs on mobile platforms.

通过使用较少参数和定量化等模型压缩技术，MobileAIBench 评估了多尺寸、定量化水平和任务，并在真实设备上测量延迟和资源消耗，旨在提供在移动平台上部署大型语言模型和多模态模型的性能和可行性洞察，加速移动 AI 研究和部署。