Recent methods for video action recognition have reached outstanding
performances on existing benchmarks. However, they tend to leverage context
such as scenes or objects instead of focusing on understanding the human action
itself. For instance, a tennis field leads to the prediction playing tennis
irrespectively of the actions performed in the video. In contrast, humans have
a more complete understanding of actions and can recognize them without
context. The best example of out-of-context actions are mimes, that people can
typically recognize despite missing relevant objects and scenes. In this paper,
we propose to benchmark action recognition methods in such absence of context
and introduce a novel dataset, Mimetics, consisting of mimed actions for a
subset of 50 classes from the Kinetics benchmark. Our experiments show that (a)
state-of-the-art 3D convolutional neural networks obtain disappointing results
on such videos, highlighting the lack of true understanding of the human
actions and (b) models leveraging body language via human pose are less prone
to context biases. In particular, we show that applying a shallow neural
network with a single temporal convolution over body pose features transferred
to the action recognition problem performs surprisingly well compared to 3D
action recognition methods.

该论文提出了一个新的数据集 Mimetics, 利用浅层神经网络在解决视频动作识别中缺乏对人类动作真正理解的问题上表现出惊人的效果，强调了人体语言通过人体姿态向行为识别问题转移的趋势。

模仿学习：探索解读人类超出环境限制的行为

Mimetics: Towards Understanding Human Actions Out of Context

We describe an extension of the DeepMind Kinetics human action dataset from
600 classes to 700 classes, where for each class there are at least 600 video
clips from different YouTube videos. This paper details the changes introduced
for this new release of the dataset, and includes a comprehensive set of
statistics as well as baseline results using the I3D neural network
architecture.

本文介绍了 DeepMind Kinetics 人类行为数据集从 600 类扩展到 700 类的详细更改，并使用 I3D 神经网络架构提供了基线结果和全面的统计数据。

Kinetics-700 人类动作数据集简明注释

A Short Note on the Kinetics-700 Human Action Dataset

In this paper, we propose a generative model which learns the relationship
between language and human action in order to generate a human action sequence
given a sentence describing human behavior. The proposed generative model is a
generative adversarial network (GAN), which is based on the sequence to
sequence (SEQ2SEQ) model. Using the proposed generative network, we can
synthesize various actions for a robot or a virtual agent using a text encoder
recurrent neural network (RNN) and an action decoder RNN. The proposed
generative network is trained from 29,770 pairs of actions and sentence
annotations extracted from MSR-Video-to-Text (MSR-VTT), a large-scale video
dataset. We demonstrate that the network can generate human-like actions which
can be transferred to a Baxter robot, such that the robot performs an action
based on a provided sentence. Results show that the proposed generative network
correctly models the relationship between language and action and can generate
a diverse set of actions from the same sentence.

本文提出了一种基于生成对抗网络 (GAN) 的生成模型，通过学习自然语言和人类行为之间的关系，在给定人类行为描述的句子的情况下生成一个人类行为序列；使用这个模型，可以使用文本编码器 RNN 和行为解码器 RNN 为机器人或虚拟代理人合成各种操作， 并通过大规模视频数据集 MSR-Video-to-Text (MSR-VTT) 进行训练，生成能够转移到 Baxter 机器人上的人类操作。结果表明，该生成对抗网络正确地建立了语言和行为之间的关系，可以从同一句子中生成不同的动作。

Text2Action: 从语言到行为的生成对抗合成

Text2Action: Generative Adversarial Synthesis from Language to Action

This work make the first attempt to generate articulated human motion
sequence from a single image. On the one hand, we utilize paired inputs
including human skeleton information as motion embedding and a single human
image as appearance reference, to generate novel motion frames, based on the
conditional GAN infrastructure. On the other hand, a triplet loss is employed
to pursue appearance-smoothness between consecutive frames. As the proposed
framework is capable of jointly exploiting the image appearance space and
articulated/kinematic motion space, it generates realistic articulated motion
sequence, in contrast to most previous video generation methods which yield
blurred motion effects. We test our model on two human action datasets
including KTH and Human3.6M, and the proposed framework generates very
promising results on both datasets.

该研究通过利用单个图像，结合人体骨骼信息、姿态运动、外观参照和 triplet loss，构建一个条件 GAN 框架，能够生成更真实的动态人体运动序列。研究数据集包括 KTH 和 Human3.6M。