In this paper, we investigate an offline reinforcement learning (RL) problem
where datasets are collected from two domains. In this scenario, having
datasets with domain labels facilitates efficient policy training. However, in
practice, the task of assigning domain labels can be resource-intensive or
infeasible at a large scale, leading to a prevalence of domain-unlabeled data.
To formalize this challenge, we introduce a novel offline RL problem setting
named Positive-Unlabeled Offline RL (PUORL), which incorporates
domain-unlabeled data. To address PUORL, we develop an offline RL algorithm
utilizing positive-unlabeled learning to predict the domain labels of
domain-unlabeled data, enabling the integration of this data into policy
training. Our experiments show the effectiveness of our method in accurately
identifying domains and learning policies that outperform baselines in the
PUORL setting, highlighting its capability to leverage domain-unlabeled data
effectively.

通过开发一种融合正负无标签学习的离线强化学习算法，该研究论文针对域未标记数据的挑战，有效地识别领域并学习优于基准的策略，以实现域未标记数据的有效利用。

跨两个领域利用无标签领域数据的离线增强学习

Leveraging Domain-Unlabeled Data in Offline Reinforcement Learning  across Two Domains

Federated learning (FL) has emerged as a powerful paradigm for learning from
decentralized data, and federated domain generalization further considers the
test dataset (target domain) is absent from the decentralized training data
(source domains). However, most existing FL methods assume that domain labels
are provided during training, and their evaluation imposes explicit constraints
on the number of domains, which must strictly match the number of clients.
Because of the underutilization of numerous edge devices and additional
cross-client domain annotations in the real world, such restrictions may be
impractical and involve potential privacy leaks. In this paper, we propose an
efficient and novel approach, called Disentangled Prompt Tuning (DiPrompT), a
method that tackles the above restrictions by learning adaptive prompts for
domain generalization in a distributed manner. Specifically, we first design
two types of prompts, i.e., global prompt to capture general knowledge across
all clients and domain prompts to capture domain-specific knowledge. They
eliminate the restriction on the one-to-one mapping between source domains and
local clients. Furthermore, a dynamic query metric is introduced to
automatically search the suitable domain label for each sample, which includes
two-substep text-image alignments based on prompt tuning without
labor-intensive annotation. Extensive experiments on multiple datasets
demonstrate that our DiPrompT achieves superior domain generalization
performance over state-of-the-art FL methods when domain labels are not
provided, and even outperforms many centralized learning methods using domain
labels.

通过学习自适应提示，采用分布式学习的方式，提出了一种高效而新颖的方法 DiPrompT，解决了现有联邦学习方法中对域标签的限制问题，实现了优秀的域泛化性能和超越中心化学习方法的结果。