Video object segmentation approaches primarily rely on large-scale
pixel-accurate human-annotated datasets for model development. In Dense Video
Object Segmentation (DVOS) scenarios, each video frame encompasses hundreds of
small, dense, and partially occluded objects. Accordingly, the labor-intensive
manual annotation of even a single frame often takes hours, which hinders the
development of DVOS for many applications. Furthermore, in videos with dense
patterns, following a large number of objects that move in different directions
poses additional challenges. To address these challenges, we proposed a
semi-self-supervised spatiotemporal approach for DVOS utilizing a
diffusion-based method through multi-task learning. Emulating real videos'
optical flow and simulating their motion, we developed a methodology to
synthesize computationally annotated videos that can be used for training DVOS
models; The model performance was further improved by utilizing weakly labeled
(computationally generated but imprecise) data. To demonstrate the utility and
efficacy of the proposed approach, we developed DVOS models for wheat head
segmentation of handheld and drone-captured videos, capturing wheat crops in
fields of different locations across various growth stages, spanning from
heading to maturity. Despite using only a few manually annotated video frames,
the proposed approach yielded high-performing models, achieving a Dice score of
0.82 when tested on a drone-captured external test set. While we showed the
efficacy of the proposed approach for wheat head segmentation, its application
can be extended to other crops or DVOS in other domains, such as crowd analysis
or microscopic image analysis.

使用多任务学习的扩散方法，我们发展了一种半自监督时空方法，通过模拟真实视频的光流和运动来合成可用于训练 DVOS 模型的计算机注释视频；在小麦头部分割方面，我们提出的方法表现出较高的性能，当在无人机捕捉的外部测试集上进行测试时，Dice 分数达到了 0.82。

DVOS: 自我监督的密集纹理视频目标分割

DVOS: Self-Supervised Dense-Pattern Video Object Segmentation

Deep learning models have been used for a variety of image processing tasks.
However, most of these models are developed through supervised learning
approaches, which rely heavily on the availability of large-scale annotated
datasets. Developing such datasets is tedious and expensive. In the absence of
an annotated dataset, synthetic data can be used for model development;
however, due to the substantial differences between simulated and real data, a
phenomenon referred to as domain gap, the resulting models often underperform
when applied to real data. In this research, we aim to address this challenge
by first computationally simulating a large-scale annotated dataset and then
using a generative adversarial network (GAN) to fill the gap between simulated
and real images. This approach results in a synthetic dataset that can be
effectively utilized to train a deep-learning model. Using this approach, we
developed a realistic annotated synthetic dataset for wheat head segmentation.
This dataset was then used to develop a deep-learning model for semantic
segmentation. The resulting model achieved a Dice score of 83.4\% on an
internal dataset and Dice scores of 79.6% and 83.6% on two external Global
Wheat Head Detection datasets. While we proposed this approach in the context
of wheat head segmentation, it can be generalized to other crop types or, more
broadly, to images with dense, repeated patterns such as those found in
cellular imagery.

使用深度学习模型结合生成对抗网络填补模拟数据与真实数据之间的差距，开发了逼真的、用于小麦苞片分割的合成数据集，并利用该数据集训练深度学习模型，取得了高达 83.4% 的 Dice 分数在内部数据集以及 79.6% 和 83.6% 的 Dice 分数在两个外部全局小麦苞头检测数据集上。该方法不仅适用于小麦苞片分割，还可以推广到其他作物类型或者具有重复、密集模式的图像，例如细胞成像。