We consider the problem of generating valid and small prediction sets by sampling outputs (e.g., software code and natural language text) from a black-box deep generative model for a given input (e.g., textual prompt). The validity of a prediction set is determined by a user-defined binary admissibility function depending on the target application. For example, requiring at least one program in the set to pass all test cases in code generation application. To address this problem, we develop a simple and effective conformal inference algorithm referred to as Generative Prediction Sets (GPS). Given a set of calibration examples and black-box access to a deep generative model, GPS can generate prediction sets with provable guarantees. The key insight behind GPS is to exploit the inherent structure within the distribution over the minimum number of samples needed to obtain an admissible output to develop a simple conformal regression approach over the minimum number of samples. Experiments on multiple datasets for code and math word problems using different large language models demonstrate the efficacy of GPS over state-of-the-art methods.

本研究解决了从深度生成模型中生成有效且小的预测集的问题，通过引入了一种称为生成预测集（GPS）的新颖方法。GPS利用样本分布的内在结构，结合简单的符合回归算法，可以生成具有可证明保证的预测集。实验表明，GPS在代码生成和数学词题等应用中优于现有最先进的方法，展示了其显著的有效性。

通过推广回归将深度生成模型的符合预测集减少到