A key characteristic of deep recommendation models is the immense memory
requirements of their embedding tables. These embedding tables can often reach
hundreds of gigabytes which increases hardware requirements and training cost.
A common technique to reduce model size is to hash all of the categorical
variable identifiers (ids) into a smaller space. This hashing reduces the
number of unique representations that must be stored in the embedding table;
thus decreasing its size. However, this approach introduces collisions between
semantically dissimilar ids that degrade model quality. We introduce an
alternative approach, Learned Hash Functions, which instead learns a new
mapping function that encourages collisions between semantically similar ids.
We derive this learned mapping from historical data and embedding access
patterns. We experiment with this technique on a production model and find that
a mapping informed by the combination of access frequency and a learned low
dimension embedding is the most effective. We demonstrate a small improvement
relative to the hashing trick and other collision related compression
techniques. This is ongoing work that explores the impact of categorical id
collisions on recommendation model quality and how those collisions may be
controlled to improve model performance.

本研究旨在改进深度推荐模型中嵌入表的巨大存储要求，提出了一种名为 Learned Hash Functions 的新方法，它从历史数据和嵌入访问模式中学习新的映射函数，并通过在语义上相似的 id 之间引入碰撞来减小嵌入表的大小，实验表明，该方法比散列技术和其他压缩技术都要更有效。