We introduce a novel LLM based solution design approach that utilizes combinatorial optimization and sampling. Specifically, a set of factors that influence the quality of the solution are identified. They typically include factors that represent prompt types, LLM inputs alternatives, and parameters governing the generation and design alternatives. Identifying the factors that govern the LLM solution quality enables the infusion of subject matter expert knowledge. Next, a set of interactions between the factors are defined and combinatorial optimization is used to create a small subset $P$ that ensures all desired interactions occur in $P$. Each element $p \in P$ is then developed into an appropriate benchmark. Applying the alternative solutions on each combination, $p \in P$ and evaluating the results facilitate the design of a high quality LLM solution pipeline. The approach is especially applicable when the design and evaluation of each benchmark in $P$ is time-consuming and involves manual steps and human evaluation. Given its efficiency the approach can also be used as a baseline to compare and validate an autoML approach that searches over the factors governing the solution.

通过使用基于组合优化和采样的新型LLM解决方案设计方法，鉴别影响解决方案质量的一系列因素，包括表示提示类型、LLM输入替代方案和生成及设计替代方案的参数。定义这些因素之间的交互作用并使用组合优化创建确保所有期望交互作用发生的小子集P，然后将P中的每个元素p开发为适当的基准，并通过应用替代方案并评估结果来促进高质量LLM解决方案流程的设计，这一方法尤其适用于每个基准的设计和评估耗时且涉及手动步骤和人工评估的情况，并且由于其高效性，该方法也可作为与解决方案主导因素进行搜索的AutoML方法进行比较和验证的基准。

利用组合优化方法设计高质量LLM解决方案