Textual information of data is of vital importance for data mining and
feature engineering. However, existing methods focus on learning the data
structures and overlook the textual information along with the data.
Consequently, they waste this valuable resource and miss out on the deeper data
relationships embedded within the texts. In this paper, we introduce
Text-Informed Feature Generation (TIFG), a novel LLM-based text-informed
feature generation framework. TIFG utilizes the textual information to generate
features by retrieving possible relevant features within external knowledge
with Retrieval Augmented Generation (RAG) technology. In this approach, the
TIFG can generate new explainable features to enrich the feature space and
further mine feature relationships. We design the TIFG to be an automated
framework that continuously optimizes the feature generation process, adapts to
new data inputs, and improves downstream task performance over iterations. A
broad range of experiments in various downstream tasks showcases that our
approach can generate high-quality and meaningful features, and is
significantly superior to existing methods.

通过利用文本信息进行特征生成，本论文介绍了一种基于语言模型的新型文本信息特征生成框架，通过使用检索增强生成（RAG）技术检索可能相关的特征并生成新的可解释特征，丰富特征空间并进一步挖掘特征关系。实验证明，该方法能够生成高质量且有意义的特征，并且明显优于现有方法。