Instance segmentation, a cornerstone task in computer vision, has
wide-ranging applications in diverse industries. The advent of deep learning
and artificial intelligence has underscored the criticality of training
effective models, particularly in data-scarce scenarios - a concern that
resonates in both academic and industrial circles. A significant impediment in
this domain is the resource-intensive nature of procuring high-quality,
annotated data for instance segmentation, a hurdle that amplifies the challenge
of developing robust models under resource constraints. In this context, the
strategic integration of a visual prior into the training dataset emerges as a
potential solution to enhance congruity with the testing data distribution,
consequently reducing the dependency on computational resources and the need
for highly complex models. However, effectively embedding a visual prior into
the learning process remains a complex endeavor. Addressing this challenge, we
introduce the MISS (Memory-efficient Instance Segmentation System) framework.
MISS leverages visual inductive prior flow propagation, integrating intrinsic
prior knowledge from the Synergy-basketball dataset at various stages: data
preprocessing, augmentation, training, and inference. Our empirical evaluations
underscore the efficacy of MISS, demonstrating commendable performance in
scenarios characterized by limited data availability and memory constraints.

通过将视觉先验知识与训练数据集相结合，MISS（Memory-efficient Instance Segmentation System）框架在数据稀缺和内存限制的情况下展现出出色的性能。

MISS：基于视觉归纳先验流传播的内存高效实例分割框架

MISS: Memory-efficient Instance Segmentation Framework By Visual  Inductive Priors Flow Propagation

Various stuff and things in visual data possess specific traits, which can be
learned by deep neural networks and are implicitly represented as the visual
prior, \emph{e.g.,} object location and shape, in the model. Such prior
potentially impacts many vision tasks. For example, in conditional image
synthesis, spatial conditions failing to adhere to the prior can result in
visually inaccurate synthetic results. This work aims to explicitly learn the
visual prior and enable the customization of sampling. Inspired by advances in
language modeling, we propose to learn Visual prior via Generative
Pre-Training, dubbed VisorGPT. By discretizing visual locations of objects,
\emph{e.g.,} bounding boxes, human pose, and instance masks, into sequences,
\our~can model visual prior through likelihood maximization. Besides, prompt
engineering is investigated to unify various visual locations and enable
customized sampling of sequential outputs from the learned prior. Experimental
results demonstrate that \our~can effectively model the visual prior, which can
be employed for many vision tasks, such as customizing accurate human pose for
conditional image synthesis models like ControlNet. Code will be released at
this https URL

本文旨在学习视觉先验，并通过生成预训练学习 Visual prior，以最大化可能性建模对象位置和形状等先验信息，进而能够在许多视觉任务中应用。实验结果表明，可用于多种视觉任务，例如控制网络中的条件图像合成模型，并提供代码供参考。