BriefGPT.xyz
Oct, 2023
推荐模型扩展时的嵌入器溃缩问题研究
On the Embedding Collapse when Scaling up Recommendation Models
HTML
PDF
Xingzhuo Guo, Junwei Pan, Ximei Wang, Baixu Chen, Jie Jiang...
TL;DR
通过对扩大模型的嵌入层进行实证和理论分析,研究人员发现嵌入坍塌现象是限制大规模推荐模型可扩展性的关键问题,并提出一种多嵌入设计来捕捉不同模式并减少崩溃,从而为各种推荐模型提供一致的扩展性。
Abstract
Recent advances in
deep foundation models
have led to a promising trend of developing large
recommendation models
to leverage vast amounts of available data. However, we experiment to scale up existing
→