BriefGPT.xyz
Oct, 2023
公私梯度耦合可证明地改善优化
Coupling public and private gradient provably helps optimization
HTML
PDF
Ruixuan Liu, Zhiqi Bu, Yu-xiang Wang, Sheng Zha, George Karypis
TL;DR
通过优化公共数据和私有数据的梯度加权线性组合，本研究分析了梯度联合的最佳权重和超参数对于非凸损失函数收敛性的加速及对语言和视觉基准的影响，为梯度联合的最优权重选择提供了指导。
Abstract
The success of
large neural networks
is crucially determined by the availability of data. It has been observed that training only on a small amount of
public data
, or privately on the abundant
→