Imitation learning seeks to circumvent the difficulty in designing proper
reward functions for training agents by utilizing expert behavior. With
environments modeled as Markov Decision Processes (MDP), most of the existing
imitation algorithms are contingent on the availability of expert
demonstrations in the same MDP as the one in which a new imitation policy is to
be learned. In this paper, we study the problem of how to imitate tasks when
there exist discrepancies between the expert and agent MDP. These discrepancies
across domains could include differing dynamics, viewpoint, or morphology; we
present a novel framework to learn correspondences across such domains.
Importantly, in contrast to prior works, we use unpaired and unaligned
trajectories containing only states in the expert domain, to learn this
correspondence. We utilize a cycle-consistency constraint on both the state
space and a domain agnostic latent space to do this. In addition, we enforce
consistency on the temporal position of states via a normalized position
estimator function, to align the trajectories across the two domains. Once this
correspondence is found, we can directly transfer the demonstrations on one
domain to the other and use it for imitation. Experiments across a wide variety
of challenging domains demonstrate the efficacy of our approach.

本篇研究针对专家行为与训练代理之间的差异，提出了一种基于无配对无对齐的轨迹，以及循环一致性限制的框架，来学习对应关系以解决领域差异的问题，并通过实验证明了该方法的有效性。

跨域观测下的模仿学习

Cross-domain Imitation from Observations

Reconstructing the 3D model of a physical object typically requires us to
align the depth scans obtained from different camera poses into the same
coordinate system. Solutions to this global alignment problem usually proceed
in two steps. The first step estimates relative transformations between pairs
of scans using an off-the-shelf technique. Due to limited information presented
between pairs of scans, the resulting relative transformations are generally
noisy. The second step then jointly optimizes the relative transformations
among all input depth scans. A natural constraint used in this step is the
cycle-consistency constraint, which allows us to prune incorrect relative
transformations by detecting inconsistent cycles. The performance of such
approaches, however, heavily relies on the quality of the input relative
transformations. Instead of merely using the relative transformations as the
input to perform transformation synchronization, we propose to use a neural
network to learn the weights associated with each relative transformation. Our
approach alternates between transformation synchronization using weighted
relative transformations and predicting new weights of the input relative
transformations using a neural network. We demonstrate the usefulness of this
approach across a wide range of datasets.

本研究提出使用神经网络学习各种相对变换之间的权重来提高图像对齐的质量，在众多数据集中证明了该方法的有效性。