Instruction-tuning can be substantially optimized through enhanced diversity,
resulting in models capable of handling a broader spectrum of tasks. However,
existing data employed for such tuning often exhibit an inadequate coverage of
individual domains, limiting the scope for nuanced comprehension and
interactions within these areas. To address this deficiency, we propose
Explore-Instruct, a novel approach to enhance the data coverage to be used in
domain-specific instruction-tuning through active exploration via Large
Language Models (LLMs). Built upon representative domain use cases,
Explore-Instruct explores a multitude of variations or possibilities by
implementing a search algorithm to obtain diversified and domain-focused
instruction-tuning data. Our data-centric analysis validates the effectiveness
of this proposed approach in improving domain-specific instruction coverage.
Moreover, our model's performance demonstrates considerable advancements over
multiple baselines, including those utilizing domain-specific data enhancement.
Our findings offer a promising opportunity to improve instruction coverage,
especially in domain-specific contexts, thereby advancing the development of
adaptable language models. Our code, model weights, and data are public at
https://github.com/fanqiwan/Explore-Instruct.

通过数据探索实现领域指导调整的数据覆盖增强方法，提高指导模型的颗粒化理解和交流能力，并在多个基准测试中取得显著进展，为改善特定领域中的指导覆盖提供了有希望的机会，推进可适应性语言模型的发展。