Prompted weak supervision (PromptedWS) applies pre-trained large language
models (LLMs) as the basis for labeling functions (LFs) in a weak supervision
framework to obtain large labeled datasets. We further extend the use of LLMs
in the loop to address one of the key challenges in weak supervision: learning
the statistical dependency structure among supervision sources. In this work,
we ask the LLM how similar are these prompted LFs. We propose a Structure
Refining Module, a simple yet effective first approach based on the
similarities of the prompts by taking advantage of the intrinsic structure in
the embedding space. At the core of Structure Refining Module are Labeling
Function Removal (LaRe) and Correlation Structure Generation (CosGen). Compared
to previous methods that learn the dependencies from weak labels, our method
finds the dependencies which are intrinsic to the LFs and less dependent on the
data. We show that our Structure Refining Module improves the PromptedWS
pipeline by up to 12.7 points on the benchmark tasks. We also explore the
trade-offs between efficiency and performance with comprehensive ablation
experiments and analysis. Code for this project can be found in
this https URL

利用大型预训练语言模型作为弱监督框架中标记函数的基础，通过扩展语言模型在循环中的应用来解决弱监督的主要挑战之一：学习监督来源之间的统计依赖结构。我们提出了一种结构细化模块的简单而有效的方法，通过利用嵌入空间中的内在结构，通过询问语言模型来评估标记函数之间的相似度。与从弱标签中学习依赖关系的先前方法相比，我们的方法找到了与标记函数本身相关的依赖关系，而不太依赖于数据。我们证明了该结构细化模块可以在基准任务上将 PromptedWS 管道的性能提升高达 12.7 个百分点，并通过全面的消融实验和分析探索了效率和性能之间的权衡。