With the increasing popularity of recommendation systems (RecSys), the demand
for compute resources in datacenters has surged. However, the model-wise
resource allocation employed in current RecSys model serving architectures
falls short in effectively utilizing resources, leading to sub-optimal total
cost of ownership. We propose ElasticRec, a model serving architecture for
RecSys providing resource elasticity and high memory efficiency. ElasticRec is
based on a microservice-based software architecture for fine-grained resource
allocation, tailored to the heterogeneous resource demands of RecSys.
Additionally, ElasticRec achieves high memory efficiency via our utility-based
resource allocation. Overall, ElasticRec achieves an average 3.3x reduction in
memory allocation size and 8.1x increase in memory utility, resulting in an
average 1.6x reduction in deployment cost compared to state-of-the-art RecSys
inference serving system.

ElasticRec 是一种模型服务架构，为推荐系统提供资源弹性性和高内存效率。相较于现有推荐系统预测服务系统，ElasticRec 通过微服务的软件架构和基于效用的资源分配实现了平均内存分配大小的 3.3 倍减少、内存效用的 8.1 倍增加，从而降低了 1.6 倍的部署成本。