Large-scale language models like ChatGPT and GPT-4 have gained attention for
their impressive conversational and generative capabilities. However, the
creation of supervised paired question-answering data for instruction tuning
presents formidable challenges. This endeavor necessitates substantial human
effort for data annotation and wrestles with issues concerning data quality,
diversity, accuracy, and other related factors. To overcome these obstacles, we
introduce an innovative framework named Self-QA, which replaces the traditional
practice of human-written instruction seeds with a vast amount of unsupervised
knowledge, enabling the model to generate a larger quantity of correct and
domain-specific instruction data. The effectiveness of our proposed method is
demonstrated through experiments conducted on unsupervised corpora from various
domains.

本文介绍了一种称为 Self-QA 的创新框架，利用大量无监督知识代替传统的人工撰写指导文件种子，从而生成更多正确和特定于领域的指导数据，以克服创建用于指导调整的监督配对问答数据所面临的挑战。