Learning to manipulate 3D objects in an interactive environment has been a
challenging problem in Reinforcement Learning (RL). In particular, it is hard
to train a policy that can generalize over objects with different semantic
categories, diverse shape geometry and versatile functionality. Recently, the
technique of visual affordance has shown great prospects in providing
object-centric information priors with effective actionable semantics. As such,
an effective policy can be trained to open a door by knowing how to exert force
on the handle. However, to learn the affordance, it often requires
human-defined action primitives, which limits the range of applicable tasks. In
this study, we take advantage of visual affordance by using the contact
information generated during the RL training process to predict contact maps of
interest. Such contact prediction process then leads to an end-to-end
affordance learning framework that can generalize over different types of
manipulation tasks. Surprisingly, the effectiveness of such framework holds
even under the multi-stage and the multi-agent scenarios. We tested our method
on eight types of manipulation tasks. Results showed that our methods
outperform baseline algorithms, including visual-based affordance methods and
RL methods, by a large margin on the success rate. The demonstration can be
found at this https URL.

本研究使用视觉契合技术，通过预测关键点的接触信息，提出了一种端到端的控制策略学习框架，可以实现在不同类型的操纵任务中的泛化。结果显示，我们的算法在成功率上大大优于基线算法，包括基于视觉契合和强化学习方法。

机器人操作的端到端操作能力学习

End-to-End Affordance Learning for Robotic Manipulation

Predicted inter-chain residue-residue contacts can be used to build the
quaternary structure of protein complexes from scratch. However, only a small
number of methods have been developed to reconstruct protein quaternary
structures using predicted inter-chain contacts. Here, we present an
agent-based self-learning method based on deep reinforcement learning
(DRLComplex) to build protein complex structures using inter-chain contacts as
distance constraints. We rigorously tested DRLComplex on two standard datasets
of homodimeric and heterodimeric protein complexes (i.e., the CASP-CAPRI
homodimer and Std_32 heterodimer datasets) using both true and predicted
interchain contacts as inputs. Utilizing true contacts as input, DRLComplex
achieved high average TM-scores of 0.9895 and 0.9881 and a low average
interface RMSD (I_RMSD) of 0.2197 and 0.92 on the two datasets, respectively.
When predicted contacts are used, the method achieves TM-scores of 0.73 and
0.76 for homodimers and heterodimers, respectively. Our experiments find that
the accuracy of reconstructed quaternary structures depends on the accuracy of
the contact predictions. Compared to other optimization methods for
reconstructing quaternary structures from inter-chain contacts, DRLComplex
performs similar to an advanced gradient descent method and better than a
Markov Chain Monte Carlo simulation method and a simulated annealing-based
method, validating the effectiveness of DRLComplex for quaternary
reconstruction of protein complexes.

利用预测的蛋白质间接触来构建蛋白质复合物的四级结构，本研究介绍了一种基于深度强化学习的自学习方法（DRLComplex），并在两个标准数据集上进行了测试，结果表明 DRLComplex 在使用真实接触信息时，可以实现高平均 TM 分数和低平均界面 RMSD，并且相对于其他优化方法，它的表现比较好，验证了它在蛋白质复合物的四级结构重构中的有效性。