An open challenge in recent machine learning is about how to improve the
reasoning capability of large language models (LLMs) in a black-box setting,
i.e., without access to detailed information such as output token
probabilities. Existing approaches either rely on accessibility (which is often
unrealistic) or involve significantly increased train- and inference-time
costs. This paper addresses those limitations or shortcomings by proposing a
novel approach, namely CoBB (Correct for improving QA reasoning of Black-Box
LLMs). It uses a trained adaptation model to perform a seq2seq mapping from the
often-imperfect reasonings of the original black-box LLM to the correct or
improved reasonings. Specifically, the adaptation model is initialized with a
relatively small open-source LLM and adapted over a collection of sub-sampled
training pairs. To select the representative pairs of correct and incorrect
reasonings, we formulated the dataset construction as an optimization problem
that minimizes the statistical divergence between the sampled subset and the
entire collection, and solved it via a genetic algorithm. We then train the
adaptation model over the sampled pairs by contrasting the likelihoods of
correct and incorrect reasonings. Our experimental results demonstrate that
CoBB significantly improves reasoning accuracy across various QA benchmarks,
compared to the best-performing adaptation baselines.

该论文提出了一种名为 CoBB 的新方法，通过使用一个训练过的适应模型来将原始黑盒 LLM 的通常不完美的推理转化为正确或改进的推理，以显著提高各种 QA 基准的推理准确性。