Recommendation systems have shown great potential to solve the information
explosion problem and enhance user experience in various online applications,
which recently present two emerging trends: (i) Collaboration: single-sided
model trained on-cloud (separate learning) to the device-cloud collaborative
recommendation (collaborative learning). (ii) Real-time Dynamic: the network
parameters are the same across all the instances (static model) to adaptive
network parameters generation conditioned on the real-time instances (dynamic
model). The aforementioned two trends enable the device-cloud collaborative and
dynamic recommendation, which deeply exploits the recommendation pattern among
cloud-device data and efficiently characterizes different instances with
different underlying distributions based on the cost of frequent device-cloud
communication. Despite promising, we argue that most of the communications are
unnecessary to request the new parameters of the recommendation system on the
cloud since the on-device data distribution are not always changing. To
alleviate this issue, we designed a Intelligent DEvice-Cloud PArameter Request
ModeL (IDEAL) that can be deployed on the device to calculate the request
revenue with low resource consumption, so as to ensure the adaptive
device-cloud communication with high revenue. We envision a new device
intelligence learning task to implement IDEAL by detecting the data
out-of-domain. Moreover, we map the user's real-time behavior to a normal
distribution, the uncertainty is calculated by the multi-sampling outputs to
measure the generalization ability of the device model to the current user
behavior. Our experimental study demonstrates IDEAL's effectiveness and
generalizability on four public benchmarks, which yield a higher efficient
device-cloud collaborative and dynamic recommendation paradigm.

提出了一种名为 IDEAL 的智能设备 - 云参数请求模型，其可以检测数据领域外的请求收入来保证高收益的自适应设备 - 云通信，利用协作学习和动态模型来实现高效的设备 - 云协作和动态推荐。实验结果表明 IDEAL 在四个公共基准测试中具有有效性和一般性。

IDEAL: 高效设备云协同动态推荐系统

IDEAL: Toward High-efficiency Device-Cloud Collaborative and Dynamic Recommendation System

A number of recent self-supervised learning methods have shown impressive
performance on image classification and other tasks. A somewhat bewildering
variety of techniques have been used, not always with a clear understanding of
the reasons for their benefits, especially when used in combination. Here we
treat the embeddings of images as point particles and consider model
optimization as a dynamic process on this system of particles. Our dynamic
model combines an attractive force for similar images, a locally dispersive
force to avoid local collapse, and a global dispersive force to achieve a
globally-homogeneous distribution of particles. The dynamic perspective
highlights the advantage of using a delayed-parameter image embedding (a la
BYOL) together with multiple views of the same image. It also uses a
purely-dynamic local dispersive force (Brownian motion) that shows improved
performance over other methods and does not require knowledge of other particle
coordinates. The method is called MSBReg which stands for (i) a Multiview
centroid loss, which applies an attractive force to pull different image view
embeddings toward their centroid, (ii) a Singular value loss, which pushes the
particle system toward spatially homogeneous density, (iii) a Brownian
diffusive loss. We evaluate downstream classification performance of MSBReg on
ImageNet as well as transfer learning tasks including fine-grained
classification, multi-class object classification, object detection, and
instance segmentation. In addition, we also show that applying our
regularization term to other methods further improves their performance and
stabilize the training by preventing a mode collapse.

本文提出一种名为 MSBReg 的动态模型，用于图像嵌入的多视角聚类。该模型结合吸引力、扩散力和纠正力，通过对相似图像的嵌入进行优化，能够在多种图像分类和迁移学习任务中表现出色，并可以用于对其他方法进行正则化以提高性能和稳定性。

嵌入动态式的自监督学习方法

An Embedding-Dynamic Approach to Self-supervised Learning

We present a probabilistic language model for time-stamped text data which
tracks the semantic evolution of individual words over time. The model
represents words and contexts by latent trajectories in an embedding space. At
each moment in time, the embedding vectors are inferred from a probabilistic
version of word2vec [Mikolov et al., 2013]. These embedding vectors are
connected in time through a latent diffusion process. We describe two scalable
variational inference algorithms--skip-gram smoothing and skip-gram
filtering--that allow us to train the model jointly over all times; thus
learning on all data while simultaneously allowing word and context vectors to
drift. Experimental results on three different corpora demonstrate that our
dynamic model infers word embedding trajectories that are more interpretable
and lead to higher predictive likelihoods than competing methods that are based
on static models trained separately on time slices.

该研究提出了一种基于概率的语言模型，可以跟踪个别单词随时间的语义演变，并通过嵌入空间中的潜在轨迹表示单词和上下文。研究发现，该动态模型推断的单词嵌入轨迹更易解释，并且具有更高的预测性可能性。