We propose a holistic approach for deploying Small Language Models (SLMs) as function-calling agents within vehicles as edge devices, offering a more flexible and robust alternative to traditional rule-based systems. By leveraging SLMs, we simplify vehicle control mechanisms and enhance the user experience. Given the in-vehicle hardware constraints, we apply state-of-the-art model compression techniques, including structured pruning, healing, and quantization, ensuring that the model fits within the resource limitations while maintaining acceptable performance. Our work focuses on optimizing a representative SLM, Microsoft's Phi-3 mini, and outlines best practices for enabling embedded models, including compression, task-specific fine-tuning, and vehicle integration. We demonstrate that, despite significant reduction in model size which removes up to 2 billion parameters from the original model, our approach preserves the model's ability to handle complex in-vehicle tasks accurately and efficiently. Furthermore, by executing the model in a lightweight runtime environment, we achieve a generation speed of 11 tokens per second, making real-time, on-device inference feasible without hardware acceleration. Our results demonstrate the potential of SLMs to transform vehicle control systems, enabling more intuitive interactions between users and their vehicles for an enhanced driving experience.

本研究针对传统规则基础系统的局限性，提出了一种整体方法，旨在将小型语言模型（SLMs）作为车载功能调用代理部署。通过优化微软的Phi-3迷你模型，采用先进的模型压缩技术，确保在硬件限制下仍能有效支持复杂的车载任务，进而提升用户体验。研究结果表明，尽管模型大小显著减少，但仍能实现高效的实时推理，表明SLMs有潜力变革车辆控制系统。

优化小型语言模型以进行车载功能调用