Recently, the growing memory demands of embedding tables in Deep Learning
Recommendation Models (DLRMs) pose great challenges for model training and
deployment. Existing embedding compression solutions cannot simultaneously meet
three key design requirements: memory efficiency, low latency, and adaptability
to dynamic data distribution. This paper presents CAFE, a Compact, Adaptive,
and Fast Embedding compression framework that addresses the above requirements.
The design philosophy of CAFE is to dynamically allocate more memory resources
to important features (called hot features), and allocate less memory to
unimportant ones. In CAFE, we propose a fast and lightweight sketch data
structure, named HotSketch, to capture feature importance and report hot
features in real time. For each reported hot feature, we assign it a unique
embedding. For the non-hot features, we allow multiple features to share one
embedding by using hash embedding technique. Guided by our design philosophy,
we further propose a multi-level hash embedding framework to optimize the
embedding tables of non-hot features. We theoretically analyze the accuracy of
HotSketch, and analyze the model convergence against deviation. Extensive
experiments show that CAFE significantly outperforms existing embedding
compression methods, yielding 3.92% and 3.68% superior testing AUC on Criteo
Kaggle dataset and CriteoTB dataset at a compression ratio of 10000x. The
source codes of CAFE are available at GitHub.

本文提出了一种紧凑、自适应和快速的嵌入式压缩框架 CAFE，旨在同时满足内存效率、低延迟和适应动态数据分布的三个关键设计要求。CAFE 采用快速轻量级的 HotSketch 数据结构实时捕捉特征重要性并报告热门特征，通过哈希嵌入技术，为非热门特征分配唯一嵌入，透过多级哈希嵌入框架对非热门特征的嵌入表进行优化，实验证明 CAFE 在 10000 倍压缩比下相比现有的嵌入式压缩方法具有 3.92% 和 3.68% 更高的测试 AUC。

CAFE：面向大规模推荐模型的紧凑、自适应、快速嵌入

CAFE: Towards Compact, Adaptive, and Fast Embedding for Large-scale  Recommendation Models

Continual Test-Time Adaptation (CTTA) aims to adapt the source model to
continually changing unlabeled target domains without access to the source
data. Existing methods mainly focus on model-based adaptation in a
self-training manner, such as predicting pseudo labels for new domain datasets.
Since pseudo labels are noisy and unreliable, these methods suffer from
catastrophic forgetting and error accumulation when dealing with dynamic data
distributions. Motivated by the prompt learning in NLP, in this paper, we
propose to learn an image-level visual domain prompt for target domains while
having the source model parameters frozen. During testing, the changing target
datasets can be adapted to the source model by reformulating the input data
with the learned visual prompts. Specifically, we devise two types of prompts,
i.e., domains-specific prompts and domains-agnostic prompts, to extract current
domain knowledge and maintain the domain-shared knowledge in the continual
adaptation. Furthermore, we design a homeostasis-based prompt adaptation
strategy to suppress domain-sensitive parameters in domain-invariant prompts to
learn domain-shared knowledge more effectively. This transition from the
model-dependent paradigm to the model-free one enables us to bypass the
catastrophic forgetting and error accumulation problems. Experiments show that
our proposed method achieves significant performance gains over
state-of-the-art methods on four widely-used benchmarks, including CIFAR-10C,
CIFAR-100C, ImageNet-C, and VLCS datasets.

本文提出了一种基于图像级视觉域提示的连续测试时间适应方法，在不访问源数据的情况下适应于持续变化的未标记目标域。该方法包括两种提示类型：域特定提示和域不可知提示，并设计了一种基于一致性的提示自适应策略。实验证明，该方法在四个广泛使用的基准测试中实现了显著的性能提高。

装扮新人：基于视觉域提示的连续测试时间适应

Decorate the Newcomers: Visual Domain Prompt for Continual Test Time  Adaptation

This paper studies the problem of expected loss minimization given a data
distribution that is dependent on the decision-maker's action and evolves
dynamically in time according to a geometric decay process. Novel algorithms
for both the information setting in which the decision-maker has a first order
gradient oracle and the setting in which they have simply a loss function
oracle are introduced. The algorithms operate on the same underlying principle:
the decision-maker repeatedly deploys a fixed decision over the length of an
epoch, thereby allowing the dynamically changing environment to sufficiently
mix before updating the decision. The iteration complexity in each of the
settings is shown to match existing rates for first and zero order stochastic
gradient methods up to logarithmic factors. The algorithms are evaluated on a
"semi-synthetic" example using real world data from the SFpark dynamic pricing
pilot study; it is shown that the announced prices result in an improvement for
the institution's objective (target occupancy), while achieving an overall
reduction in parking rates.

研究了基于体积衰减过程动态演变的决策者行动依赖数据分布下的最小化预期损失问题，提出了针对信息设置和损失函数的两种全新算法，经过实验验证可显著提高机构目标（目标占用）并降低停车费率。