We study the learning dynamics of self-predictive learning for reinforcement
learning, a family of algorithms that learn representations by minimizing the
prediction error of their own future latent representations. Despite its recent
empirical success, such algorithms have an apparent defect: trivial
representations (such as constants) minimize the prediction error, yet it is
obviously undesirable to converge to such solutions. Our central insight is
that careful designs of the optimization dynamics are critical to learning
meaningful representations. We identify that a faster paced optimization of the
predictor and semi-gradient updates on the representation, are crucial to
preventing the representation collapse. Then in an idealized setup, we show
self-predictive learning dynamics carries out spectral decomposition on the
state transition matrix, effectively capturing information of the transition
dynamics. Building on the theoretical insights, we propose bidirectional
self-predictive learning, a novel self-predictive algorithm that learns two
representations simultaneously. We examine the robustness of our theoretical
insights with a number of small-scale experiments and showcase the promise of
the novel representation learning algorithm with large-scale experiments.

本篇研究探讨了自预测学习的学习动态，通过对优化动态的设计，提出了双向自学习算法，并通过一系列实验验证了该算法的有效性。

理解强化学习中的自预测学习

Understanding Self-Predictive Learning for Reinforcement Learning

Domain adaptation for semantic image segmentation is very necessary since
manually labeling large datasets with pixel-level labels is expensive and time
consuming. Existing domain adaptation techniques either work on limited
datasets, or yield not so good performance compared with supervised learning.
In this paper, we propose a novel bidirectional learning framework for domain
adaptation of segmentation. Using the bidirectional learning, the image
translation model and the segmentation adaptation model can be learned
alternatively and promote to each other. Furthermore, we propose a
self-supervised learning algorithm to learn a better segmentation adaptation
model and in return improve the image translation model. Experiments show that
our method is superior to the state-of-the-art methods in domain adaptation of
segmentation with a big margin. The source code is available at
this https URL

本研究提出了一种基于双向学习和自我监督学习的域自适应语义图像分割框架，与现有技术相比，在大幅优化分割结果的同时，避免了对大规模数据集的手工标注。

语义分割领域自适应的双向学习

Bidirectional Learning for Domain Adaptation of Semantic Segmentation

A multilayer perceptron can behave as a generative classifier by applying
bidirectional learning (BL). It consists of training an undirected neural
network to map input to output and vice-versa; therefore it can produce a
classifier in one direction, and a generator in the opposite direction for the
same data. The learning process of BL tries to reproduce the neuroplasticity
stated in Hebbian theory using only backward propagation of errors. In this
paper, two novel learning techniques are introduced which use BL for improving
robustness to white noise static and adversarial examples. The first method is
bidirectional propagation of errors, which the error propagation occurs in
backward and forward directions. Motivated by the fact that its generative
model receives as input a constant vector per class, we introduce as a second
method the hybrid adversarial networks (HAN). Its generative model receives a
random vector as input and its training is based on generative adversarial
networks (GAN). To assess the performance of BL, we perform experiments using
several architectures with fully and convolutional layers, with and without
bias. Experimental results show that both methods improve robustness to white
noise static and adversarial examples, and even increase accuracy, but have
different behavior depending on the architecture and task, being more
beneficial to use the one or the other. Nevertheless, HAN using a convolutional
architecture with batch normalization presents outstanding robustness, reaching
state-of-the-art accuracy on adversarial examples of hand-written digits.

本文介紹了兩種使用雙向學習 (BL) 來改善韌性的新學習技術，分別為雙向誤差傳播和混合對抗性網絡（HAN）並且證明了兩者均可顯著提高模型對白噪聲靜態和對抗性樣本的韌性，並根據不同的任務和體系結構選擇不同的算法。