This study introduces GPTA, a Large Language Model assistance training
framework, that enhances the training of downstream task models via prefix
prompt. By minimizing data exposure to LLM, the framework addresses the
security and legal challenges of applying LLM in downstream task model
training. GPTA utilizes a new synergistic training approach, optimizing the
downstream models with parameter gradients and LLMs with the novel ``dialogue
gradient''. The framework not only demonstrates significant improvements in
model performance across six NLP benchmark datasets, but also reduces
overfitting in low-resource scenarios effectively. The detailed analyses
further validate that our pioneer framework provides a cost-efficient and
adaptive method for downstream task model training with LLM support.

GPTA 是一个通过预设前缀加强下游任务模型的训练的大型语言模型辅助训练框架，通过最小化对 LLM 的数据暴露，解决了在下游任务模型训练中应用 LLM 所面临的安全和法律挑战。GPTA 采用了一种新的协同训练方法，通过参数梯度优化下游模型和通过新颖的 “对话梯度” 优化 LLM。该框架不仅在六个自然语言处理基准数据集上展示出显著的模型性能提升，而且在资源稀缺场景中有效减少了过拟合。详细分析进一步验证了我们的先导框架提供了一种成本高效和适应性强的 LLM 支持下游任务模型训练的方法。