We present ComplexityNet, a streamlined language model designed for assessing
task complexity. This model predicts the likelihood of accurate output by
various language models, each with different capabilities. Our initial
application of ComplexityNet involves the Mostly Basic Python Problems (MBPP)
dataset. We pioneered the creation of the first set of labels to define task
complexity. ComplexityNet achieved a notable 79% accuracy in determining task
complexity, a significant improvement over the 34% accuracy of the original,
non fine-tuned model. Furthermore, ComplexityNet effectively reduces
computational resource usage by 90% compared to using the highest complexity
model, while maintaining a high code generation accuracy of 86.7%. This study
demonstrates that fine-tuning smaller models to categorize tasks based on their
complexity can lead to a more balanced trade-off between accuracy and
efficiency in the use of Large Language Models. Our findings suggest a
promising direction for optimizing LLM applications, especially in
resource-constrained environments.

我们提出了 ComplexityNet，这是一个精简的语言模型，专用于评估任务的复杂性。我们对 Python 问题进行了复杂性标签的创建，并且发现 ComplexityNet 在确定任务复杂性方面的准确率达到了 79％，相比于未经微调的原始模型的 34％准确率有显著提高。此外，与使用最高复杂性模型相比，ComplexityNet 有效地减少了 90％的计算资源使用，并且保持了 86.7％较高的代码生成准确性。该研究表明，在使用大型语言模型时，通过微调较小模型以基于任务复杂性进行分类可以在准确性和效率之间取得更平衡的权衡。我们的发现为优化 LLM 应用提供了有希望的方向，特别是在资源受限环境中。