This paper presents a new method for unsupervised segmentation of complex
activities from video into multiple steps, or sub-activities, without any
textual input. We propose an iterative discriminative-generative approach which
alternates between discriminatively learning the appearance of sub-activities
from the videos' visual features to sub-activity labels and generatively
modelling the temporal structure of sub-activities using a Generalized Mallows
Model. In addition, we introduce a model for background to account for frames
unrelated to the actual activities. Our approach is validated on the
challenging Breakfast Actions and Inria Instructional Videos datasets and
outperforms both unsupervised and weakly-supervised state of the art.

本文介绍了一种新的无监督视频分割方法，采用迭代鉴别生成方法，通过学习视频视觉特征并建模时序结构，将复杂活动划分为多个子活动，并引入背景模型，验证结果优于现有无监督及弱监督方法。

从视频中无监督学习和分割复杂活动

Unsupervised Learning and Segmentation of Complex Activities from Video

Understanding human activities and object affordances are two very important
skills, especially for personal robots which operate in human environments. In
this work, we consider the problem of extracting a descriptive labeling of the
sequence of sub-activities being performed by a human, and more importantly, of
their interactions with the objects in the form of associated affordances.
Given a RGB-D video, we jointly model the human activities and object
affordances as a Markov random field where the nodes represent objects and
sub-activities, and the edges represent the relationships between object
affordances, their relations with sub-activities, and their evolution over
time. We formulate the learning problem using a structural support vector
machine (SSVM) approach, where labelings over various alternate temporal
segmentations are considered as latent variables. We tested our method on a
challenging dataset comprising 120 activity videos collected from 4 subjects,
and obtained an accuracy of 79.4% for affordance, 63.4% for sub-activity and
75.0% for high-level activity labeling. We then demonstrate the use of such
descriptive labeling in performing assistive tasks by a PR2 robot.

该研究旨在利用 Markov 随机场和支持向量机算法，提取描述性标签以识别人类活动和物品仿真，并应用于机器人的辅助任务中，其准确度在 120 个视频的测试中分别为 80% (物品仿真）、64%（子活动）和 75% (高级活动）。

从 RGB-D 视频中学习人类活动和物体可支配性

Learning Human Activities and Object Affordances from RGB-D Videos

Being able to detect and recognize human activities is essential for several
applications, including personal assistive robotics. In this paper, we perform
detection and recognition of unstructured human activity in unstructured
environments. We use a RGBD sensor (Microsoft Kinect) as the input sensor, and
compute a set of features based on human pose and motion, as well as based on
image and pointcloud information. Our algorithm is based on a hierarchical
maximum entropy Markov model (MEMM), which considers a person's activity as
composed of a set of sub-activities. We infer the two-layered graph structure
using a dynamic programming approach. We test our algorithm on detecting and
recognizing twelve different activities performed by four people in different
environments, such as a kitchen, a living room, an office, etc., and achieve
good performance even when the person was not seen before in the training set.

使用 RGBD 传感器和最大熵隐马尔科夫模型识别人类活动，通过层次化建模的方法，将活动分解为多个子活动，实验结果表明在不同环境下能够识别出 12 种不同的人类活动。