Designing deep learning-based solutions is becoming a race for training deeper models with a greater number of layers. While a large-size deeper model could provide competitive accuracy, it creates a lot of logistical challenges and unreasonable resource requirements during development and deployment. This has been one of the key reasons for deep learning models not being excessively used in various production environments, especially in edge devices. There is an immediate requirement for optimizing and compressing these deep learning models, to enable on-device intelligence. In this research, we introduce a black-box framework, Deeplite Neutrino^{TM} for production-ready optimization of deep learning models. The framework provides an easy mechanism for the end-users to provide constraints such as a tolerable drop in accuracy or target size of the optimized models, to guide the whole optimization process. The framework is easy to include in an existing production pipeline and is available as a Python Package, supporting PyTorch and Tensorflow libraries. The optimization performance of the framework is shown across multiple benchmark datasets and popular deep learning models. Further, the framework is currently used in production and the results and testimonials from several clients are summarized.

该研究介绍了一种基于深度学习的黑盒优化框架，Deeplite Neutrino，旨在优化深度学习模型的大小，从而在边缘设备上实现设备上智能应用。该框架易于将其纳入现有生产流程并支持PyTorch和Tensorflow库。同时，该框架的性能经过了多个基准数据集和流行的深度学习模型的优化，且在多个客户端获得了成果和证明。

Deeplite Neutrino: 用于受限深度学习模型优化的端到端框架