Segmenting cells and tracking their motion over time is a common task in
biomedical applications. However, predicting accurate instance-wise
segmentation and cell motions from microscopy imagery remains a challenging
task. Using microstructured environments for analyzing single cells in a
constant flow of media adds additional complexity. While large-scale labeled
microscopy datasets are available, we are not aware of any large-scale dataset,
including both cells and microstructures. In this paper, we introduce the
trapped yeast cell (TYC) dataset, a novel dataset for understanding
instance-level semantics and motions of cells in microstructures. We release
$105$ dense annotated high-resolution brightfield microscopy images, including
about $19$k instance masks. We also release $261$ curated video clips composed
of $1293$ high-resolution microscopy images to facilitate unsupervised
understanding of cell motions and morphology. TYC offers ten times more
instance annotations than the previously largest dataset, including cells and
microstructures. Our effort also exceeds previous attempts in terms of
microstructure variability, resolution, complexity, and capturing device
(microscopy) variability. We facilitate a unified comparison on our novel
dataset by introducing a standardized evaluation strategy. TYC and evaluation
code are publicly available under CC BY 4.0 license.

本文介绍了用于理解微结构中细胞实例级语义和运动的全新数据集，提供了 105 个密集标注的高分辨率亮场显微镜图像和 19k 个实例遮罩，以及 261 个精选的视频剪辑，包括 1293 个高分辨率显微镜图像，便于对细胞运动和形态进行无监督理解。该数据集包括细胞和微结构的实例注释，并通过引入标准化的评估策略，提供了一个统一的比较平台。

TYC 数据集分析微结构中细胞的实例级语义和运动

The TYC Dataset for Understanding Instance-Level Semantics and Motions  of Cells in Microstructures

This paper aims to establish a generic multi-modal foundation model that has
the scalable capability to massive downstream applications in E-commerce.
Recently, large-scale vision-language pretraining approaches have achieved
remarkable advances in the general domain. However, due to the significant
differences between natural and product images, directly applying these
frameworks for modeling image-level representations to E-commerce will be
inevitably sub-optimal. To this end, we propose an instance-centric multi-modal
pretraining paradigm called ECLIP in this work. In detail, we craft a decoder
architecture that introduces a set of learnable instance queries to explicitly
aggregate instance-level semantics. Moreover, to enable the model to focus on
the desired product instance without reliance on expensive manual annotations,
two specially configured pretext tasks are further proposed. Pretrained on the
100 million E-commerce-related data, ECLIP successfully extracts more generic,
semantic-rich, and robust representations. Extensive experimental results show
that, without further fine-tuning, ECLIP surpasses existing methods by a large
margin on a broad range of downstream tasks, demonstrating the strong
transferability to real-world E-commerce applications.

本研究提出了一种实例为中心的多模态预训练范式 ECLIP，通过引入可学习实例查询的解码器架构和两个预处理任务，从 1 亿个电子商务相关数据中预训练出语义丰富、稳健的模型，进而在广泛的下游任务中超越现有方法，证明其在现实世界中的广泛可转移性。

电商大规模多模态预训练中的实例级表示学习

Learning Instance-Level Representation for Large-Scale Multi-Modal  Pretraining in E-commerce

This paper presents a unified framework for depth-aware panoptic segmentation
(DPS), which aims to reconstruct 3D scene with instance-level semantics from
one single image. Prior works address this problem by simply adding a dense
depth regression head to panoptic segmentation (PS) networks, resulting in two
independent task branches. This neglects the mutually-beneficial relations
between these two tasks, thus failing to exploit handy instance-level semantic
cues to boost depth accuracy while also producing sub-optimal depth maps. To
overcome these limitations, we propose a unified framework for the DPS task by
applying a dynamic convolution technique to both the PS and depth prediction
tasks. Specifically, instead of predicting depth for all pixels at a time, we
generate instance-specific kernels to predict depth and segmentation masks for
each instance. Moreover, leveraging the instance-wise depth estimation scheme,
we add additional instance-level depth cues to assist with supervising the
depth learning via a new depth loss. Extensive experiments on Cityscapes-DPS
and SemKITTI-DPS show the effectiveness and promise of our method. We hope our
unified solution to DPS can lead a new paradigm in this area. Code is available
at this https URL

本文提出了一种深度感知全景分割 (DPS) 的统一框架，该框架通过应用动态卷积技术来解决全景分割和深度预测任务之间的互补关系，从而实现对单张图像进行实例级语义重建，通过实例级深度估计方案，为辅助监督深度学习而添加额外的实例级深度提示，实验结果表明该方法具有实用性和前景。