Modern deep neural networks (DNNs) have greatly facilitated the development
of sequential recommender systems by achieving state-of-the-art recommendation
performance on various sequential recommendation tasks. Given a sequence of
interacted items, existing DNN-based sequential recommenders commonly embed
each item into a unique vector to support subsequent computations of the user
interest. However, due to the potentially large number of items, the
over-parameterised item embedding matrix of a sequential recommender has become
a memory bottleneck for efficient deployment in resource-constrained
environments, e.g., smartphones and other edge devices. Furthermore, we observe
that the widely-used multi-head self-attention, though being effective in
modelling sequential dependencies among items, heavily relies on redundant
attention units to fully capture both global and local item-item transition
patterns within a sequence.
In this paper, we introduce a novel lightweight self-attentive network (LSAN)
for sequential recommendation. To aggressively compress the original embedding
matrix, LSAN leverages the notion of compositional embeddings, where each item
embedding is composed by merging a group of selected base embedding vectors
derived from substantially smaller embedding matrices. Meanwhile, to account
for the intrinsic dynamics of each item, we further propose a temporal
context-aware embedding composition scheme. Besides, we develop an innovative
twin-attention network that alleviates the redundancy of the traditional
multi-head self-attention while retaining full capacity for capturing long- and
short-term (i.e., global and local) item dependencies. Comprehensive
experiments demonstrate that LSAN significantly advances the accuracy and
memory efficiency of existing sequential recommenders.

该论文介绍了一种新型轻量级自注意网络 LSAN，通过组成嵌入将原始嵌入矩阵进行压缩并提出了时态上下文感知的嵌入组合方案以解决目前序列推荐中所存在的大量参数嵌入矩阵对资源受限环境的开销过大的问题，并拥有更强的性能表现。

轻量级自注意力序列推荐

Lightweight Self-Attentive Sequential Recommendation

DNN-based frame interpolation--that generates the intermediate frames given
two consecutive frames--typically relies on heavy model architectures with a
huge number of features, preventing them from being deployed on systems with
limited resources, e.g., mobile devices. We propose a compression-driven
network design for frame interpolation (CDFI), that leverages model pruning
through sparsity-inducing optimization to significantly reduce the model size
while achieving superior performance. Concretely, we first compress the
recently proposed AdaCoF model and show that a 10X compressed AdaCoF performs
similarly as its original counterpart; then we further improve this compressed
model by introducing a multi-resolution warping module, which boosts visual
consistencies with multi-level details. As a consequence, we achieve a
significant performance gain with only a quarter in size compared with the
original AdaCoF. Moreover, our model performs favorably against other
state-of-the-arts in a broad range of datasets. Finally, the proposed
compression-driven framework is generic and can be easily transferred to other
DNN-based frame interpolation algorithm. Our source code is available at
this https URL

本文提出了一种基于模型稀疏性优化的压缩驱动网络设计方法，用于减小 DNN 图像插值模型的大小，并引入多分辨率变形模块以提高视觉一致性，相比原始模型具有更好的性能表现。

基于压缩的网络设计在帧内插值中的应用 (CDFI)

CDFI: Compression-Driven Network Design for Frame Interpolation

DNN-based cross-modal retrieval has become a research hotspot, by which users
can search results across various modalities like image and text. However,
existing methods mainly focus on the pairwise correlation and reconstruction
error of labeled data. They ignore the semantically similar and dissimilar
constraints between different modalities, and cannot take advantage of
unlabeled data. This paper proposes Cross-modal Deep Metric Learning with
Multi-task Regularization (CDMLMR), which integrates quadruplet ranking loss
and semi-supervised contrastive loss for modeling cross-modal semantic
similarity in a unified multi-task learning architecture. The quadruplet
ranking loss can model the semantically similar and dissimilar constraints to
preserve cross-modal relative similarity ranking information. The
semi-supervised contrastive loss is able to maximize the semantic similarity on
both labeled and unlabeled data. Compared to the existing methods, CDMLMR
exploits not only the similarity ranking information but also unlabeled
cross-modal data, and thus boosts cross-modal retrieval accuracy.

本文提出了 CDMLMR 方法，该方法以多任务正则化的方式融合四元组排名损失和半监督对比损失，用于建模跨模态语义相似性；与现有方法相比，该方法不仅利用了相似性排名信息，还利用了未标记的跨模态数据，从而提高了跨模态检索准确性。