Gradient-free prompt optimization methods have made significant strides in
enhancing the performance of closed-source Large Language Models (LLMs) across
a wide range of tasks. However, existing approaches make light of the
importance of high-quality prompt initialization and the identification of
effective optimization directions, thus resulting in substantial optimization
steps to obtain satisfactory performance. In this light, we aim to accelerate
prompt optimization process to tackle the challenge of low convergence rate. We
propose a dual-phase approach which starts with generating high-quality initial
prompts by adopting a well-designed meta-instruction to delve into
task-specific information, and iteratively optimize the prompts at the sentence
level, leveraging previous tuning experience to expand prompt candidates and
accept effective ones. Extensive experiments on eight datasets demonstrate the
effectiveness of our proposed method, achieving a consistent accuracy gain over
baselines with less than five optimization steps.

通过采用良好设计的元指令，我们提出了一个双重阶段的方法来加速提示优化过程，以应对低收敛速度的挑战，并在句级别上迭代优化提示，利用之前的调优经验扩展提示候选并接受有效的候选。对八个数据集进行的大量实验证明了我们提出的方法的有效性，在少于五次优化步骤的情况下与基准模型相比实现了一致的准确率提升。