The rapid rise of Language Models (LMs) has expanded their use in several
applications. Yet, due to constraints of model size, associated cost, or
proprietary restrictions, utilizing state-of-the-art (SOTA) LLMs is not always
feasible. With open, smaller LMs emerging, more applications can leverage their
capabilities, but selecting the right LM can be challenging. This work conducts
an in-depth experimental analysis of the semantic correctness of outputs of 10
smaller, open LMs across three aspects: task types, application domains and
reasoning types, using diverse prompt styles. We demonstrate that most
effective models and prompt styles vary depending on the specific requirements.
Our analysis provides a comparative assessment of LMs and prompt styles using a
proposed three-tier schema of aspects for their strategic selection based on
use-case and other constraints. We also show that if utilized appropriately,
these LMs can compete with, and sometimes outperform, SOTA LLMs like
DeepSeek-v2, GPT-3.5-Turbo, and GPT-4o.

使用十个较小、开放式的语言模型在任务类型、应用领域和推理类型等三个方面进行了深入的实验分析，比较评估了语言模型和提示样式，并且展示了这些模型在特定需求下的有效性，以及与 SOTA 语言模型的竞争能力。