Plan-and-Write is a common hierarchical approach in long-form narrative text
generation, which first creates a plan to guide the narrative writing.
Following this approach, several studies rely on simply prompting large
language models for planning, which often yields suboptimal results. In this
paper, we propose a new framework called Evaluation-guided Iterative Plan
Extraction for long-form narrative text generation (EIPE-text), which extracts
plans from the corpus of narratives and utilizes the extracted plans to
construct a better planner. EIPE-text has three stages: plan extraction,
learning, and inference. In the plan extraction stage, it iteratively extracts
and improves plans from the narrative corpus and constructs a plan corpus. We
propose a question answer (QA) based evaluation mechanism to automatically
evaluate the plans and generate detailed plan refinement instructions to guide
the iterative improvement. In the learning stage, we build a better planner by
fine-tuning with the plan corpus or in-context learning with examples in the
plan corpus. Finally, we leverage a hierarchical approach to generate long-form
narratives. We evaluate the effectiveness of EIPE-text in the domains of novels
and storytelling. Both GPT-4-based evaluations and human evaluations
demonstrate that our method can generate more coherent and relevant long-form
narratives. Our code will be released in the future.

我们提出了一个名为 EIPE-text 的新框架，通过从叙述语料库中提取计划并利用提取的计划构建更好的规划器，实现了长篇叙述文本的生成。通过基于问答（QA）的评估机制和迭代改进指导，我们证明了 EIPE-text 方法在小说和讲故事领域可以生成更连贯和相关的长篇叙述文本。