Generative Large Language Models (LLMs) stand as a revolutionary advancement
in the modern era of artificial intelligence (AI). However, directly deploying
LLMs in resource-constrained hardware, such as Internet-of-Things (IoT)
devices, is difficult due to their high computational cost. In this paper, we
propose a novel information-entropy framework for designing mobile-friendly
generative language models. Our key design paradigm is to maximize the entropy
of transformer decoders within the given computational budgets. The whole
design procedure involves solving a mathematical programming (MP) problem,
which can be done on the CPU within minutes, making it nearly zero-cost. We
evaluate our designed models, termed MeRino, across nine NLP downstream tasks,
showing their competitive performance against the state-of-the-art
autoregressive transformer models under the mobile setting. Notably, MeRino
achieves similar or better zero performance compared to the 350M parameter OPT
while being 4.9x faster on NVIDIA Jetson Nano with 5.5x reduction in model
size. Code will be made available soon.

我们提出了一种新颖的信息熵框架，用于设计面向移动设备的生成式语言模型。我们通过最大化给定计算预算内的 Transformer 解码器的熵来进行关键设计，从而在数分钟内在 CPU 上解决数学规划问题，几乎为零成本。我们评估了我们设计的模型 MeRino，在九个 NLP 下游任务中展示了它们在移动设置下与最先进的自回归 Transformer 模型竞争的性能。值得注意的是，MeRino 在 NVIDIA Jetson Nano 上的速度是 350M 参数 OPT 的 4.9 倍快，模型大小减小了 5.5 倍，但具有类似或更好的零性能。代码将很快提供。