When we humans look at a video of human-object interaction, we can not only
infer what is happening but we can even extract actionable information and
imitate those interactions. On the other hand, current recognition or geometric
approaches lack the physicality of action representation. In this paper, we
take a step towards a more physical understanding of actions. We address the
problem of inferring contact points and the physical forces from videos of
humans interacting with objects. One of the main challenges in tackling this
problem is obtaining ground-truth labels for forces. We sidestep this problem
by instead using a physics simulator for supervision. Specifically, we use a
simulator to predict effects and enforce that estimated forces must lead to the
same effect as depicted in the video. Our quantitative and qualitative results
show that (a) we can predict meaningful forces from videos whose effects lead
to accurate imitation of the motions observed, (b) by jointly optimizing for
contact point and force prediction, we can improve the performance on both
tasks in comparison to independent training, and (c) we can learn a
representation from this model that generalizes to novel objects using few shot
examples.

本文利用物理模拟器进行监督，解决了从人物和物体的视频中推断接触点和物理力量的问题，并展示了学习到的模型可通过小样本实现对新物体的泛化。

利用模拟效果学习预测物理力量：使用原力，卢克！

Use the Force, Luke! Learning to Predict Physical Forces by Simulating  Effects

We introduce Act2Vec, a general framework for learning context-based action
representation for Reinforcement Learning. Representing actions in a vector
space help reinforcement learning algorithms achieve better performance by
grouping similar actions and utilizing relations between different actions. We
show how prior knowledge of an environment can be extracted from demonstrations
and injected into action vector representations that encode natural compatible
behavior. We then use these for augmenting state representations as well as
improving function approximation of Q-values. We visualize and test action
embeddings in three domains including a drawing task, a high dimensional
navigation task, and the large action space domain of StarCraft II.

Act2Vec 是一种通用的框架，用于学习基于上下文的强化学习中的行动表示，在此基础上，该文章针对三个领域 (绘画任务、高维导航任务和星际争霸 2) 进行了可视化和测试，并展示了先前环境知识如何从示范中提取并注入到编码自然兼容行为的行动向量表示中，最终优化了 Q 值函数的近似。

动作的自然语言

The Natural Language of Actions

Most model-free reinforcement learning methods leverage state representations
(embeddings) for generalization, but either ignore structure in the space of
actions or assume the structure is provided a priori. We show how a policy can
be decomposed into a component that acts in a low-dimensional space of action
representations and a component that transforms these representations into
actual actions. These representations improve generalization over large, finite
action sets by allowing the agent to infer the outcomes of actions similar to
actions already taken. We provide an algorithm to both learn and use action
representations and provide conditions for its convergence. The efficacy of the
proposed method is demonstrated on large-scale real-world problems.

研究了在没有给定先验结构的情况下，如何基于状态表示和行为表示实现模型无关的强化学习方法，并提供了相应的算法和收敛条件。

强化学习的动作表示学习

Learning Action Representations for Reinforcement Learning

What defines an action like "kicking ball"? We argue that the true meaning of
an action lies in the change or transformation an action brings to the
environment. In this paper, we propose a novel representation for actions by
modeling an action as a transformation which changes the state of the
environment before the action happens (precondition) to the state after the
action (effect). Motivated by recent advancements of video representation using
deep learning, we design a Siamese network which models the action as a
transformation on a high-level feature space. We show that our model gives
improvements on standard action recognition datasets including UCF101 and
HMDB51. More importantly, our approach is able to generalize beyond learned
action categories and shows significant performance improvement on
cross-category generalization on our new ACT dataset.

本文提出将动作作为转换模型的一种新方法，并通过设计 Siamese 网络来学习动作的高层次特征，实现了对于视频数据的动作识别，同时在 ACT 数据集上显示出了重要的跨类别泛化性能改进。