In the development of science, accurate and reproducible documentation of the
experimental process is crucial. Automatic recognition of the actions in
experiments from videos would help experimenters by complementing the recording
of experiments. Towards this goal, we propose FineBio, a new fine-grained video
dataset of people performing biological experiments. The dataset consists of
multi-view videos of 32 participants performing mock biological experiments
with a total duration of 14.5 hours. One experiment forms a hierarchical
structure, where a protocol consists of several steps, each further decomposed
into a set of atomic operations. The uniqueness of biological experiments is
that while they require strict adherence to steps described in each protocol,
there is freedom in the order of atomic operations. We provide hierarchical
annotation on protocols, steps, atomic operations, object locations, and their
manipulation states, providing new challenges for structured activity
understanding and hand-object interaction recognition. To find out challenges
on activity understanding in biological experiments, we introduce baseline
models and results on four different tasks, including (i) step segmentation,
(ii) atomic operation detection (iii) object detection, and (iv)
manipulated/affected object detection. Dataset and code are available from
this https URL

科学发展中，准确且可重复记录实验过程至关重要。本文提出 FineBio，一种新的细粒度视频数据集，用于研究人员进行生物实验。该数据集具有 32 个参与者执行模拟生物实验的多视角视频，总时长为 14.5 小时。通过对协议、步骤、原子操作、对象位置和其操作状态进行层次化标注，提供了构造活动理解和手 — 物体交互识别的新挑战。通过引入基线模型，并在步骤分割、原子操作检测、对象检测和操作 / 影响对象检测等四个不同任务上的结果，探究了在生物实验中的活动理解挑战。数据集和代码可以通过该 https URL 获得。

FineBio：生物实验的细粒度视频数据集及层次化注释

FineBio: A Fine-Grained Video Dataset of Biological Experiments with  Hierarchical Annotation

Temporal action localization (TAL) is an important and challenging problem in
video understanding. However, most existing TAL benchmarks are built upon the
coarse granularity of action classes, which exhibits two major limitations in
this task. First, coarse-level actions can make the localization models overfit
in high-level context information, and ignore the atomic action details in the
video. Second, the coarse action classes often lead to the ambiguous
annotations of temporal boundaries, which are inappropriate for temporal action
localization. To tackle these problems, we develop a novel large-scale and
fine-grained video dataset, coined as FineAction, for temporal action
localization. In total, FineAction contains 103K temporal instances of 106
action categories, annotated in 17K untrimmed videos. Compared to the existing
TAL datasets, our FineAction takes distinct characteristics of fine action
classes with rich diversity, dense annotations of multiple instances, and
co-occurring actions of different classes, which introduces new opportunities
and challenges for temporal action localization. To benchmark FineAction, we
systematically investigate the performance of several popular temporal
localization methods on it, and deeply analyze the influence of fine-grained
instances in temporal action localization. As a minor contribution, we present
a simple baseline approach for handling the fine-grained action detection,
which achieves an mAP of 13.17% on our FineAction. We believe that FineAction
can advance research of temporal action localization and beyond.

开发了一个新的大规模、细粒度视频数据集 FineAction，共包含 106 个动作类别的 103k 个时间实例，在 17k 个未修剪的视频中进行了注释，用于对时间动作定位进行基准测试，并深入分析了细粒度实例对时间动作定位的影响。