Obtaining stakeholders' diverse experiences and opinions about current policy
in a timely manner is crucial for policymakers to identify strengths and gaps
in resource allocation, thereby supporting effective policy design and
implementation. However, manually coding even moderately sized interview texts
or open-ended survey responses from stakeholders can often be labor-intensive
and time-consuming. This study explores the integration of Large Language
Models (LLMs)--like GPT-4--with human expertise to enhance text analysis of
stakeholder interviews regarding K-12 education policy within one U.S. state.
Employing a mixed-methods approach, human experts developed a codebook and
coding processes as informed by domain knowledge and unsupervised topic
modeling results. They then designed prompts to guide GPT-4 analysis and
iteratively evaluate different prompts' performances. This combined
human-computer method enabled nuanced thematic and sentiment analysis. Results
reveal that while GPT-4 thematic coding aligned with human coding by 77.89% at
specific themes, expanding to broader themes increased congruence to 96.02%,
surpassing traditional Natural Language Processing (NLP) methods by over 25%.
Additionally, GPT-4 is more closely matched to expert sentiment analysis than
lexicon-based methods. Findings from quantitative measures and qualitative
reviews underscore the complementary roles of human domain expertise and
automated analysis as LLMs offer new perspectives and coding consistency. The
human-computer interactive approach enhances efficiency, validity, and
interpretability of educational policy research.

该研究通过将人类专业知识与大型语言模型（如 GPT-4）相结合，探索了人与计算机相互配合进行教育政策研究中的文本分析。结果显示，GPT-4 在特定主题上与人类编码的统一性达到 77.89％，在更广泛的主题上达到 96.02％，超过传统自然语言处理方法 25％以上，并且 GPT-4 更接近专家的情感分析。这种人与计算机相互配合的方法提高了教育政策研究的效率、有效性和可解释性。