Leveraging ``chain-of-thought (CoT)'' reasoning to elicit rapid and precise
responses from large language models (LLMs) is rapidly attracting research
interest. A notable challenge here is how to design or select optimal prompts.
The process of prompt selection relies on trial and error, involving continuous
adjustments and combinations of input prompts by users based on the
corresponding new responses generated from LLMs. Furthermore, minimal research
has been conducted to explore how LLMs employ the mathematical problem-solving
capabilities learned from user interactions to address issues in narrative
writing. To improve interpretability and explore the balance principle between
generality and personalization under a multi-domain CoT prompt selection
scenario, we propose the Federated Logic rule learning approach (FedLogic). We
introduce a theoretical formalization and interactive emulation of the
multi-domain CoT prompt selection dilemma in the context of federated LLMs. We
cast the problem of joint probability modeling as a bilevel program, where the
CoT prompt selection intricacy can be likened to a fuzzy score-based rule
selection with the LLMs function as rule generators. FedLogic solves this
problem through variational expectation maximization (V-EM). In addition, we
incorporate two KL-divergence constraints within this probabilistic modeling
framework to surmount the intricacies of managing extensive search spaces and
accomplishing cross-domain personalization of CoTs. To the best of our
knowledge, FedLogic is the first interpretable and principled federated
multi-domain CoT prompt selection approach for LLMs.

使用 “连续思维” 推理从大型语言模型中提取快速而精确的响应吸引了研究兴趣，其中关键挑战在于设计或选择最佳提示，并且该论文介绍了一种名为 FedLogic 的可解释的联邦多领域连续思维提示选择方法，以提高解释性和探索广义性与个性化之间的平衡原则。