In the development of science, accurate and reproducible documentation of the experimental process is crucial. Automatic recognition of the actions in experiments from videos would help experimenters by complementing the recording of experiments. Towards this goal, we propose FineBio, a new fine-grained video dataset of people performing biological experiments. The dataset consists of multi-view videos of 32 participants performing mock biological experiments with a total duration of 14.5 hours. One experiment forms a hierarchical structure, where a protocol consists of several steps, each further decomposed into a set of atomic operations. The uniqueness of biological experiments is that while they require strict adherence to steps described in each protocol, there is freedom in the order of atomic operations. We provide hierarchical annotation on protocols, steps, atomic operations, object locations, and their manipulation states, providing new challenges for structured activity understanding and hand-object interaction recognition. To find out challenges on activity understanding in biological experiments, we introduce baseline models and results on four different tasks, including (i) step segmentation, (ii) atomic operation detection (iii) object detection, and (iv) manipulated/affected object detection. Dataset and code are available from https://github.com/aistairc/FineBio.

科学发展中，准确且可重复记录实验过程至关重要。本文提出FineBio，一种新的细粒度视频数据集，用于研究人员进行生物实验。该数据集具有32个参与者执行模拟生物实验的多视角视频，总时长为14.5小时。通过对协议、步骤、原子操作、对象位置和其操作状态进行层次化标注，提供了构造活动理解和手—物体交互识别的新挑战。通过引入基线模型，并在步骤分割、原子操作检测、对象检测和操作/影响对象检测等四个不同任务上的结果，探究了在生物实验中的活动理解挑战。数据集和代码可以通过该https URL获得。

FineBio：生物实验的细粒度视频数据集及层次化注释