In this work, we conduct an assessment of the optimization capabilities of
LLMs across various tasks and data sizes. Each of these tasks corresponds to
unique optimization domains, and LLMs are required to execute these tasks with
interactive prompting. That is, in each optimization step, the LLM generates
new solutions from the past generated solutions with their values, and then the
new solutions are evaluated and considered in the next optimization step.
Additionally, we introduce three distinct metrics for a comprehensive
assessment of task performance from various perspectives. These metrics offer
the advantage of being applicable for evaluating LLM performance across a broad
spectrum of optimization tasks and are less sensitive to variations in test
samples. By applying these metrics, we observe that LLMs exhibit strong
optimization capabilities when dealing with small-sized samples. However, their
performance is significantly influenced by factors like data size and values,
underscoring the importance of further research in the domain of optimization
tasks for LLMs.

评估 LLM 在各种任务和数据大小上的优化能力，并引入了三个不同的指标来全面评估任务性能。通过应用这些指标，我们观察到 LLM 在处理小规模样本时表现出很强的优化能力，但其性能受到数据大小和值等因素的显著影响，强调了对 LLM 的优化任务领域进行进一步研究的重要性。